Open AI's popular chatbot ChatGPT may have aced the Wharton Master of Business Administration (MBA) examination, however, according to a recent report, it failed the Union Public Service Commission (UPSC) exam held in India to train and recruit civil servants.
The chatbot that was launched in November 2022, has created ripples through the academic, creative, and technological industry, to the extent where it has now raised questions of eliminating people's jobs. A popular belief goes that ChatGPT has all the answers.
However, the UPSC exam proved otherwise.
While the chatbot easily cleared some of the most difficult exams around the world, including the Master of Business Administration (MBA) programme at the University of Pennsylvania’s Wharton School to the US medical exam and further the US bar exam for law, the chatbot failed to clear even the prelims to the UPSC examination.
For the uninitiated, the Union Public Service Commission conducts various examinations to recruit eligible candidates for the Indian Government.
The latest attempt for the chatbot was made possible by Analytics India Magazine (AIM). When the chatbot was asked, “Do you think you can pass the prelim exam for UPSC?", it knew that it was going to be difficult, the magazine noted. The chatbot had to say that it cannot give a “definite answer" on whether it can pass the exam or not.
AIM asked ChatGPT all 100 questions from Question Paper 1 (Set A) from the UPSC Prelims 2022. ChatGPT could only answer 54 of them correctly which was a surprise for the magazine company. The cutoff for the general category candidates was 87.54 which suggests that ChatGPT couldn’t clear the UPSC examination.
Notably, ChatGPT's knowledge is limited till September 2021. Therefore it failed to answer current affairs questions for India. But that is not it!
The believed-to-be-know-it-all chatbot failed to answer Geography, History, and Economy questions from the UPSC paper as well. Further in some multiple-choice questions, the chatbot created an extra option that wasn’t provided in the original question and labelled it “None of the above".
