Rare Disease Differential Diagnosis with Large Language Models at Scale: From Abdominal Actinomycosis to Wilson's Disease
Large language models (LLMs) struggle with accurate rare-disease diagnosis in primary care due to data scarcity and inherent biases toward common conditions. Method: This paper proposes RareScale—a novel framework that synergistically integrates expert systems with black-box LLMs (e.g., GPT-4o) to synthesize high-quality, rare-disease–specific consultation dialogues; constructs a lightweight candidate predictor to supply LLMs with precise prior inputs; and designs a multi-stage diagnostic pipeline unifying rule-based reasoning and supervised fine-tuning to dynamically balance rare- and common-disease diagnostic capabilities. Contributions/Results: Evaluated on 575 rare diseases, RareScale achieves an 17.1% absolute improvement in Top-5 diagnostic accuracy and 88.8% candidate generation accuracy—significantly surpassing pure LLM baselines and overcoming key performance bottlenecks in long-tail disease recognition.