MedEinst: Benchmarking the Einstellung Effect in Medical LLMs through Counterfactual Differential Diagnosis

2026-01-10

arXiv.org

AI Summary

This study addresses the susceptibility of medical large language models (LLMs) to cognitive biases—particularly set effects—in atypical clinical cases, where reliance on statistical shortcuts often overrides patient-specific evidence, leading to misdiagnosis. Existing evaluation benchmarks fail to capture such diagnostic biases effectively. To this end, the authors introduce MedEinst, the first counterfactual clinical benchmark comprising 5,383 paired control and “trap” cases, and propose the Bias Trap Rate metric to quantify diagnostic bias. They further design the ECR-Agent framework, which integrates dual-path dynamic causal reasoning (DCI), a three-layer causal graph (associational, interventional, and counterfactual), evidence auditing, and a critique-driven graph memory evolution mechanism to enable auditable and iterative evidence-based reasoning. Experiments reveal that while mainstream medical LLMs achieve high accuracy, they exhibit substantial Bias Trap Rates; in contrast, ECR-Agent significantly mitigates this bias and enhances diagnostic robustness on atypical cases.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Despite achieving high accuracy on medical benchmarks, LLMs exhibit the Einstellung Effect in clinical diagnosis--relying on statistical shortcuts rather than patient-specific evidence, causing misdiagnosis in atypical cases. Existing benchmarks fail to detect this critical failure mode. We introduce MedEinst, a counterfactual benchmark with 5,383 paired clinical cases across 49 diseases. Each pair contains a control case and a"trap"case with altered discriminative evidence that flips the diagnosis. We measure susceptibility via Bias Trap Rate--probability of misdiagnosing traps despite correctly diagnosing controls. Extensive Evaluation of 17 LLMs shows frontier models achieve high baseline accuracy but severe bias trap rates. Thus, we propose ECR-Agent, aligning LLM reasoning with Evidence-Based Medicine standard via two components: (1) Dynamic Causal Inference (DCI) performs structured reasoning through dual-pathway perception, dynamic causal graph reasoning across three levels (association, intervention, counterfactual), and evidence audit for final diagnosis; (2) Critic-Driven Graph and Memory Evolution (CGME) iteratively refines the system by storing validated reasoning paths in an exemplar base and consolidating disease-specific knowledge into evolving illness graphs. Source code is to be released.

Problem

• Einstellung Effect
• medical LLMs
• diagnostic bias
• counterfactual reasoning
• clinical diagnosis

Innovation

• Einstellung Effect
• Counterfactual Benchmarking
• Dynamic Causal Inference
• Evidence-Based Medicine
• Medical LLMs