PPOL Divisional Seminar - LLMs in Court
Supporting the below United Nations Sustainable Development Goals:支持以下聯合國可持續發展目標：支持以下联合国可持续发展目标：
Large language models (LLMs) are increasingly integrated into judicial processes, assisting with research, summarization, and drafting. However, their unverified use threatens core principles, including accountability, transparency, and ultimately public trust in the judiciary. Our study demonstrates interpretive divergences across leading LLMs (GPT‑4, Claude, Gemini, LLaMA) on benchmarked legal tasks, revealing systematic vulnerabilities that we term omission, injection, and framing. Through controlled perturbations, semantic analyses, and cross-model evaluations on CaseHOLD and LexGLUE, we quantify semantic drift and show that even small variations in prompts or model design can materially alter legal reasoning. To address these risks, we situate our research within broader scholarship in law and the social sciences on judicial behavior. Building on this foundation, we propose a governance framework designed to ensure that LLM adoption strengthens, rather than compromises, the fairness and legitimacy of judicial decision-making, thereby safeguarding public trust in the judiciary. This paper was presented at one of the workshops held in conjunction with IJCAI 2025 in Montreal, where it received a Best Paper Award and is forthcoming in a special issue of Springer Lecture Notes in Artificial Intelligence (LNAI).
Dr. Sara's research focuses on AI policy, AI-related harms, and AI in dispute resolution. Her work has been cited several times by the Court of Justice of the EU. Prior to joining the University of Macau, she served as a policy researcher at the British Institute of International and Comparative Law (BIICL), where she led policy studies for the European Commission. Sara holds a Ph.D. from the European University Institute (EUI), a Master’s Degree from the University of Paris 1, Panthéon-Sorbonne and is admitted to the Paris Bar.
Dr. Djallel’s research focuses on online machine learning and autonomous systems that learn and adapt in uncertain environments. Over the past decade at IBM, he has worked on reinforcement learning, brain modeling, and ethical AI systems with enhanced trustworthiness. His earlier work includes developing recommender systems at Nomalys, dialogue systems at Orange Labs, and biomedical data analysis at BC Cancer Agency. His research has resulted in over 100 publications. He serves on program committees for more than 20 conferences in AI and machine learning.