MSc(DDM) Expert Sharing Seminar Series - LLM-as-a-Judge and Persona Prompting: Emerging Issues in AI Evaluation and Trust

We are excited to have Mr. Hinz SHUM, Senior Manager (Global AI Model Validation and Governance) of Manulife, to share the topic “LLM-as-a-Judge and Persona Prompting: Emerging Issues in AI Evaluation and Trust”.

The seminar is open to HKUST School of Science students. 

Details are as follows.

Date: 29 April 2026 (Wednesday)
Time: Starts At 11:00 AM HKT
Speaker: Mr, Hinz SHUM, Senior Manager, Global AI Model Validation and Governance, Manulife
Host: Prof. Sai-ping LI, Adjunct Professor, Department of Physics, HKUST
Via Zoom: Please contact the MSc DDM Program Office at mscddm@ust.hk for information on Zoom.

Introduction

The "LLM-as-a-Judge" paradigm reframes large language models from generators to evaluators, capable of scoring, ranking, and critiquing outputs under complex, qualitative criteria. However, emerging evidence shows that evaluation behaviour is not neutral: mechanisms that improve alignment, especially persona prompting, can simultaneously degrade factual accuracy and reasoning performance. This seminar examines LLM-based evaluation as a probabilistic, context-dependent process, and explores its implications for reliability, system design, and high-stakes decision-making environments.

Seminar Topic and Overview:

This seminar provides a high-level introduction to two emerging themes in modern artificial intelligence: LLM-as-a-Judge and persona prompting. Large language models are increasingly used not only to generate content, but also to support the development and improvement of AI systems by evaluating, ranking, and reviewing model outputs. At the same time, assigning models different roles or personas, such as expert, advisor, regulator, or teacher, can significantly influence how they reason, communicate, and make judgments. Together, these developments are reshaping how AI systems are designed, assessed, and trusted.

The session will begin by comparing traditional evaluation methods with newer model-based approaches, showing how AI systems can assist in assessing other AI systems at scale. It will then explore the growing use of persona prompting, where assigning models roles such as expert, advisor, regulator, or teacher may improve tone, structure, or task focus, but can also introduce overconfidence, bias, or responses that sound persuasive without being fully reliable.

The seminar will further discuss the interaction between these two themes: if an AI system is used as an evaluator, how might its assigned persona influence the fairness, consistency, or usefulness of its judgments? Recent developments in adaptive frameworks will also be introduced, where different evaluation styles or judging approaches may be applied depending on context and task type.

Finally, the discussion will consider practical applications in areas such as finance, risk management, and governance, where accuracy, accountability, and human oversight remain especially important. Designed for students interested in AI, analytics, and real-world decision systems, the seminar aims to provide both conceptual understanding and practical perspective on how AI evaluation and AI behaviour design may shape the next generation of intelligent systems.

About the speaker: 

Mr. Hinz SHUM is a Senior Manager in Global AI Model Validation and Governance at Manulife, where he oversees the governance and responsible use of all AI models across Asia. His work focuses on ensuring that AI systems are deployed with appropriate risk controls, due diligence, and alignment with business and regulatory expectations. He is also an active algorithmic trading practitioner and has served as a mentor to students at various local universities.

Prior to joining Manulife, Mr. Shum held a Manager role at the Hong Kong Monetary Authority, including in the Banking Supervisory Technology Division, where he supported supervisory initiatives relating to the banking sector’s use of AI and emerging technologies. He also served as a Data Scientist in the Research Department, focusing on the application of advanced analytics and new AI developments to operational use cases.

Earlier in his career, he worked as a Statistician specialising in Data Science within the Hong Kong Government, where he contributed to analytics-driven public sector initiatives, drawing on the knowledge and training gained from the HKUST MSc in Data-Driven Modelling Programme.

*Remarks:

1. This is an online seminar.

2. The seminar is open to HKUST School of Science students. 

3. Please note that the event will be recorded. By accessing the virtual meeting, participants consent to the recording of audio, screen content, and video, which may be used by the HKUST MSc in Data-Driven Modelling for news coverage, promotional and advertising purposes, and inclusion on websites, social media platforms, or other communication channels.