Humans v machine : The AFSA-UP moot court experiment with ChatGPT 4.0 (April 2024)

Abstract

The legal profession is being challenged to harness the predictive capabilities of AI as a fundamental tool reshaping the future of law. New technologies require a reimagining of the essence of legal services. Against this background, the University of Pretoria, Faculty of Law was invited to use ChatGPT to generate an arbitration award for the 2024 Johannesburg Arbitration Week (JAW) Young AFSA Moot. The AI-generated award was then juxtaposed with an award produced by a panel of expert human arbitrators. The primary objective of this experiment was to gauge the effectiveness and reliability of AI in producing legal judgments, a domain traditionally dominated by human expertise. This article discusses the findings and implications of this experiment, illuminating the evolving legal landscape and AI’s profound impact on the future of legal practice. It discusses the concept of generative AI and its application in law. It then explains and reflects on the arbitration award experiment. It goes on to analyse key issues as produced by ChatGPT and the human arbitrators. The article suggests several aspects that must be considered in the context of generative AI and legal practice and education. It concludes by emphasising that the integration of AI in legal practice promises to revolutionise the field, offering unprecedented efficiency and accuracy. However, for this transformation to be effective and ethically sound, both legal education and practice must evolve substantially.