Evaluating Scaffolding-Oriented Multi-Agent Large Language Model System for Clinical Interview Training

2026-09-09

AI Summary

为解决临床访谈培训资源密集和难以扩展的问题，研究开发了一个基于多代理大型语言模型的AI标准化患者培训平台，并通过随机对照实验验证了其在提高医学生沟通、同理心及病史采集行为方面的有效性。

Abstract

Clinical education must prepare medical students to conduct safe and coherent patient interviews under conditions of uncertainty. Traditional standardized patient (SP) training is resource-intensive and difficult to scale. We developed a scaffolding-oriented multi-agent Large Language Model (LLM) AI Standardized Patient (AI-SP) training platform1. The system includes a patient agent for simulated dialog, a tutor agent providing Socratic prompts without disclosing diagnostic information, and a turn-level evaluator agent that monitors clinical progress without revealing summative scores. In a randomized controlled study (N = 100 medical students), participants were assigned to either a multi-agent (MA) scaffolding condition or a control condition. All students completed two learning sessions under their assigned condition followed by an examination conducted in a patient only environment. Performance was assessed using a standardized Objective Structured Clinical Examination (OSCE) based rubric. While no significant difference was observed in final diagnostic accuracy between groups, the multi-agent AI standardized patient system improved final examination scores compared to the control group utilizing structured progressive information disclosure; the most substantial and consistent improvements were observed in communication, the expression of empathy, and specific history-taking behaviors. These findings suggest that specialized LLM agents enhance the process quality of simulated clinical interviews without artificially inflating examination outcomes. To support future research, we release a multi-expert annotated dataset comprising transcripts, checklist annotations, turn-level evaluations, and OSCE-aligned scoring outcomes. This resource aims to facilitate the development of pedagogically grounded AI-SP systems and advance research on AI-supported clinical reasoning training.

Problem

• clinical education
• medical students
• patient interviews
• uncertainty
• standardized patient training

Innovation

• scaffolding-oriented multi-agent
• Large Language Model (LLM)
• AI Standardized Patient (AI-SP)
• structured progressive information disclosure
• clinical interview training