Large Language Model Aided Birt-Hogg-Dube Syndrome Diagnosis with Multimodal Retrieval-Augmented Generation

2025-11-24

AI Summary

Addressing two key challenges in Birt-Hogg-Dubé syndrome (BHD) diagnosis—scarcity of clinical samples and high inter-class similarity among diffuse cystic lung diseases (DCLDs) leading to poor generalizability of deep learning models and elevated hallucination risks in large language models—this paper proposes BHD-RAG, a multimodal retrieval-augmented generation framework. BHD-RAG constructs the first BHD-specific multimodal corpus, incorporates radiological priors, and introduces a traceable CT feature retrieval module based on cosine similarity. It further integrates an image captioning agent with a clinical precedent fusion mechanism to enable evidence-driven diagnostic reasoning. Evaluated on a four-class DCLD dataset, BHD-RAG achieves significantly improved diagnostic accuracy and generates outputs highly consistent with expert assessments, effectively mitigating domain knowledge gaps and hallucination issues inherent in large foundation models.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Deep learning methods face dual challenges of limited clinical samples and low inter-class differentiation among Diffuse Cystic Lung Diseases (DCLDs) in advancing Birt-Hogg-Dube syndrome (BHD) diagnosis via Computed Tomography (CT) imaging. While Multimodal Large Language Models (MLLMs) demonstrate diagnostic potential fo such rare diseases, the absence of domain-specific knowledge and referable radiological features intensify hallucination risks. To address this problem, we propose BHD-RAG, a multimodal retrieval-augmented generation framework that integrates DCLD-specific expertise and clinical precedents with MLLMs to improve BHD diagnostic accuracy. BHDRAG employs: (1) a specialized agent generating imaging manifestation descriptions of CT images to construct a multimodal corpus of DCLDs cases. (2) a cosine similarity-based retriever pinpointing relevant imagedescription pairs for query images, and (3) an MLLM synthesizing retrieved evidence with imaging data for diagnosis. BHD-RAG is validated on the dataset involving four types of DCLDs, achieving superior accuracy and generating evidence-based descriptions closely aligned with expert insights.

Problem

• Diagnosing Birt-Hogg-Dube syndrome from CT images with limited clinical samples
• Reducing hallucinations in multimodal LLMs for rare disease diagnosis
• Improving diagnostic accuracy by integrating domain knowledge and clinical precedents

Innovation

• Multimodal retrieval-augmented generation framework for diagnosis
• Specialized agent constructs DCLDs imaging manifestation corpus
• Cosine similarity retrieves relevant image-description pairs