By: Kendall Casey
On March 14, 2023, the newest version of OpenAI passed the Uniform Bar Exam (“UBE”) with scores that would place it in the 90th percentile of test takers. This comes just two months after the previous version of the software attempted the bar exam in January, not answering particular portions and scoring poorly on portions that it did take. This rapid leap forward demonstrates the importance for professionals and lawyers alike to learn about this new technology and how it can be incorporated into legal work.
The Artificial Intelligence company, OpenAI, was founded in 2015 with the goal of developing AI and machine learning tools for video games and other recreational purposes. The company introduced the concept of a Generative Pre-Trained Transformer (“GPT”) as a “neutral network, or a machine learning model, created to function like a human brain, and it’s trained on input (large data sets to produce outputs (i.e., answers to users’ questions).” In 2022, the company released ChatGPT, which was an advanced chatbot that could answer questions to a wide range of questions.
In January 2023, GPT-3.5 was tested on the multiple-choice part of the bar exam. It was able to answer about 50% of the questions correctly with a meager placement at the 10th percentile of human-test takers. Further, it only received a passing score in one area. However, in March 2023, GPT-4 answered 76% of the questions correctly, placing it in the 90th percentile of human-test takers. GPT-4s would receive a score of 297 on the UBE, which overcomes the passing score by a large amount, far surpassing Arizona’s required score of 273, which is the highest threshold in the country.
This rapid development of GPT is one that professionals and lawyers alike should take note of. GPT has the potential to increase workload capacity and efficiency. For example, passing the bar required not only ordinary English, but also the use of complex “legalese,” which is difficult even for humans. GPT is able to read text, understand, classify, and analyze it, all of which are key components to the practice of law. This has been described as a new age of the practice of law and can provide unprecedented value for both lawyers and clients.
Like any other novel technology, there are concerns with GPT technology and using it for legal work. For example, GPT models have been known to “hallucinate,” by generating completely false, but convincing, responses to questions. However, the newest version of GPT, GPT-4, hallucinates much less. While GPT did answer some questions incorrectly on the exam, they were similar to wrong answers made by human-test takers. However, hallucinations still happen with GPT-4 and users need to be aware and on the lookout for them.
GPT is no longer a technology that will be used far in the future. It is something that can and should be used now. GPT can make the practice of law more efficient and allow lawyers to focus on more pressing issues. The rapid transformation between models demonstrates the new age of law and the importance of adopting new technology that will provide value to lawyers and their clients. However, lawyers should not be concerned that AI systems will take over their jobs. When asked whether GPT could pass the bar exam, it responded as follows:
As an AI language model, I do not have the ability to take the bar exam or receive a score on it. While I can process and analyze information related to a legal fact pattern, I do not have the capacity to take an exam or be graded on it. It is important to note that passing the bar exam is a rigorous and challenging process that requires years of study and preparation. While I have been trained on a vast array of legal topics and can provide insight on legal issues, I am not a substitute for a licensed attorney who has completed the necessary education and training to practice law.
Student Bio: Kendall Casey is a second-year law student at Suffolk University Law School. She is a staff writer for the Journal of High Technology Law. Kendall received a Bachelor’s of Science in Business Management and a Bachelor’s of Arts in Spanish Studies from the University of Delaware.
Disclaimer: The views expressed in this blog are the views of the author alone and do not represent the views of JHTL or Suffolk University Law School.
