Human-LLM Dialogue Improves Diagnostic Accuracy in Emergency Care

2026-05-08

AI Summary

This study addresses the challenge of making rapid and accurate clinical decisions in emergency settings under high uncertainty, where empirical evidence on the effectiveness of large language models (LLMs) in real-world physician workflows remains scarce. The authors propose MedSyn, a system enabling clinicians to engage in multi-turn, conversational queries with an LLM using only patient chief complaints, leveraging full electronic health records to support diagnostic reasoning. Evaluated in a real emergency department, MedSyn significantly improves diagnostic accuracy—particularly among resident physicians—with absolute gains of 0.145 in exact-match correctness, 0.156 in standardized diagnostic accuracy, and 0.138 in F1 score for complex cases. The intervention also enhances diagnostic consistency across clinicians of varying experience levels and reveals distinct interaction strategies employed by physicians at different career stages.

Abstract

Clinical decision-making in emergency medicine demands rapid, accurate diagnoses under uncertainty. Despite benchmark progress, evidence for LLMs as interactive aids in live physician workflows remains sparse. MedSyn lets physicians iteratively query an LLM provided with the full clinical record while initially viewing only the chief complaint. Seven physicians (three seniors, four residents) completed baseline and AI-assisted sessions across 52 MIMIC-IV cases stratified by difficulty. Blinded evaluation showed residents' Hard-case correctness rose from 0.589 to 0.734; difficulty-standardised completely-correct rates confirmed a medium effect (Δ = 0.092; p = 0.071; d = 0.47). Automated metrics corroborated these gains: standardised any-match accuracy improved by 0.156 (p < 0.0001), and residents showed the largest F1 gain (Δ = 0.138; p < 0.0001). Dialogue analysis revealed expertise-dependent strategies (seniors asked targeted, hypothesis-driven questions; residents relied on broader queries) and cross-expertise concordance increased (Δ = 0.145; p < 0.0001). Interactive LLM support meaningfully enhances diagnostic reasoning.

Problem

• emergency care
• diagnostic accuracy
• human-LLM dialogue
• clinical decision-making
• interactive AI assistance

Innovation

• interactive LLM
• diagnostic reasoning
• clinical decision support
• human-AI collaboration
• emergency medicine