Benchmarking Clinical Decision Pathway Adherence in Large Language Models

2026-08-27

AI Summary

为解决医疗大语言模型遵循临床决策路径（CDP）的问题，通过构建MEGA-CDP基准测试，评估模型生成符合指南的CDP的能力。

Abstract

Following clinical decision pathways (CDPs) defined by clinical practice guidelines is essential for safe and reliable medical decision-making. However, existing medical large language model (LLM) benchmarks mainly evaluate final-answer accuracy, providing limited evaluation of models' ability to adhere to guidelines. To address this gap, we introduce MEGA-CDP, a benchmark for evaluating whether medical LLMs can generate guideline-adherent CDPs using provided guidelines as references. MEGA-CDP is constructed from 2,274 English and Chinese clinical practice guidelines through a guideline-to-case pipeline, yielding 42,353 clinical cases with explicit reference CDPs. It supports both single-turn vignette and multi-turn interactive settings, and introduces a CDP-oriented evaluation framework for measuring pathway consistency. Experiments on 16 representative LLMs show that reliable clinical decision support remains challenging for current models, demonstrating the need for CDP-oriented evaluation and the value of MEGA-CDP for advancing guideline adherence in medical LLMs.

Problem

• Clinical Decision Pathways
• Large Language Models
• Guideline Adherence

Innovation

• MEGA-CDP
• Clinical Decision Pathways
• Large Language Models
• Guideline Adherence
• Evaluation Framework