ChatGPT passes law school exam - where were you when the robots took our jobs?


Icon Extra
29 Nov 2022

A chatbot powered by reams of data from the internet has passed exams at a US law school after writing essays on topics ranging from constitutional law to taxation and torts.

ChatGPT from OpenAI, a US company that this week got a massive injection of cash from Microsoft, uses artificial intelligence (AI) to generate streams of text from simple prompts.

The results have been so good that educators have warned it could lead to widespread cheating and even signal the end of traditional classroom teaching methods.

Jonathan Choi, a professor at Minnesota University Law School, gave ChatGPT the same test faced by students, consisting of 95 multiple-choice questions and 12 essay questions.

In a white paper titled "ChatGPT goes to law school" published on Monday, he and his coauthors reported that the bot scored a C+ overall.

While this was enough for a pass, the bot was near the bottom of the class in most subjects and "bombed" at multiple-choice questions involving mathematics.

"In writing essays, ChatGPT displayed a strong grasp of basic legal rules and had consistently solid organization and composition," the authors wrote.

But the bot "often struggled to spot issues when given an open-ended prompt, a core skill on law school exams".

Officials in New York and other jurisdictions have banned the use of ChatGPT in schools, but Choi suggested it could be a valuable teaching aide.

"Overall, ChatGPT wasn't a great law student acting alone," he wrote on Twitter.

"But we expect that collaborating with humans, language models like ChatGPT would be very useful to law students taking exams and to practicing lawyers."

And playing down the possibility of cheating, he wrote in reply to another Twitter user that two out of three markers had spotted the bot-written paper.

"(They) had a hunch and their hunch was right, because ChatGPT had perfect grammar and was somewhat repetitive," Choi wrote.

This is actually a pretty big wakeup call. Within 10 years this technology will probably be good enough to stand in for the average human in 95% of knowledge worker fields. There was a big to-do about what happens when robots start taking all the menial jobs like long haul trucking...but just wait til 90% of white collar work is obviated. Our society will collapse when no one has a way to contribute value.

Night Sky

30 Dec 2022
Don't worry, automation and machines will do the work for us and we will leave in peace and harmony - Most people, just before the industrial revolution.


Did you like my Glasses, Snake?
21 Jun 2022
More cheating

There's a pretty simple solution for this. Test time? Phone goes in a locker, we catch you with an electronic device during a test, you fail.

Edit: this is what you get for not reading the articles lol. I guess they are using them to write essays not cheat during tests, which is likely much harder to police.
  • Informative
Reactions: Night Sky