Domain-Adapted Small Language Models for Reliable Clinical Triage

2026-04-29

AI Summary

This study addresses the high variability in free-text triage notes within emergency departments, which contributes to inconsistent Emergency Severity Index (ESI) acuity assignments and compromises clinical accuracy and efficiency. To mitigate this issue, the authors propose an institution-tailored small language model (SLM) decision support system based on Qwen2.5-7B, fine-tuned through domain adaptation using expert-annotated data and a silver-standard pediatric triage dataset. The work systematically evaluates multiple prompting strategies and demonstrates that clinical-summary prompts yield the best performance. The fine-tuned model significantly reduces both ESI assignment inconsistency and clinically significant error rates, outperforming existing open-source SLMs as well as closed-source large models such as GPT-4o, while achieving a balanced integration of accuracy, stability, and patient privacy preservation.

Abstract

Accurate and consistent Emergency Severity Index (ESI) assignment remains a persistent challenge in emergency departments, where highly variable free-text triage documentation contributes to mistriage and workflow inefficiencies. This study evaluates whether open-source small language models (SLMs) can serve as reliable, privacy-preserving decision-support tools for clinical triage. We systematically compared multiple SLMs across diverse prompting pipelines and found that clinical vignettes, concise summaries of triage narratives, yielded the most accurate predictions. The SLM, Qwen2.5-7B, demonstrated the strongest balance of accuracy, stability, and computational efficiency. Through large-scale domain adaptation using expert-curated and silver-standard pediatric triage data, fine-tuned Qwen2.5-7B models substantially reduced discordance and clinically significant errors, outperforming all baseline SLMs and advanced proprietary large language models (LLMs, e.g., GPT-4o). These findings highlight the feasibility of institution-specific SLMs for reliable, privacy-preserving ESI decision support and underscore the importance of targeted fine-tuning over more complex inference strategies.

Problem

• Emergency Severity Index
• clinical triage
• mistriage
• workflow inefficiencies
• free-text triage documentation

Innovation

• domain adaptation
• small language models
• clinical triage
• Emergency Severity Index
• fine-tuning