An Agentic System for Rare Disease Diagnosis with Traceable Reasoning

2025-06-25

AI Summary

Rare disease diagnosis suffers from high misdiagnosis rates and low interpretability due to pronounced clinical heterogeneity, low prevalence, and limited clinician awareness. To address these challenges, we propose DeepRare—the first large language model (LLM)-based agent system specifically designed for rare disease diagnosis. DeepRare adopts a modular, scalable architecture that integrates Human Phenotype Ontology (HPO)-driven phenotypic analysis, multimodal input processing, and over 40 biomedical tools; it further incorporates long-term memory and real-time knowledge updating to enable traceable, verifiable, chain-of-thought reasoning. Evaluated on eight benchmark datasets covering 1,013 rare diseases, DeepRare achieves 100% diagnostic accuracy, an HPO Recall@1 of 57.18%—23.79 percentage points higher than the second-best method—and a 95.40% expert acceptance rate for its reasoning chains. These results demonstrate substantial improvements in diagnostic accuracy, transparency, and clinical trustworthiness.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Machine Learning: Large Multimodal Models (LMMs)
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Rare diseases collectively affect over 300 million individuals worldwide, yet timely and accurate diagnosis remains a pervasive challenge. This is largely due to their clinical heterogeneity, low individual prevalence, and the limited familiarity most clinicians have with rare conditions. Here, we introduce DeepRare, the first rare disease diagnosis agentic system powered by a large language model (LLM), capable of processing heterogeneous clinical inputs. The system generates ranked diagnostic hypotheses for rare diseases, each accompanied by a transparent chain of reasoning that links intermediate analytic steps to verifiable medical evidence. DeepRare comprises three key components: a central host with a long-term memory module; specialized agent servers responsible for domain-specific analytical tasks integrating over 40 specialized tools and web-scale, up-to-date medical knowledge sources, ensuring access to the most current clinical information. This modular and scalable design enables complex diagnostic reasoning while maintaining traceability and adaptability. We evaluate DeepRare on eight datasets. The system demonstrates exceptional diagnostic performance among 2,919 diseases, achieving 100% accuracy for 1013 diseases. In HPO-based evaluations, DeepRare significantly outperforms other 15 methods, like traditional bioinformatics diagnostic tools, LLMs, and other agentic systems, achieving an average Recall@1 score of 57.18% and surpassing the second-best method (Reasoning LLM) by a substantial margin of 23.79 percentage points. For multi-modal input scenarios, DeepRare achieves 70.60% at Recall@1 compared to Exomiser's 53.20% in 109 cases. Manual verification of reasoning chains by clinical experts achieves 95.40% agreements. Furthermore, the DeepRare system has been implemented as a user-friendly web application http://raredx.cn/doctor.

Problem

• Develops AI system for rare disease diagnosis
• Improves diagnostic accuracy with traceable reasoning
• Integrates diverse medical knowledge and tools

Innovation

• LLM-powered rare disease diagnosis agent
• Modular design with specialized agent servers
• Transparent reasoning chain with medical evidence