CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation

2024-06-17

AI Summary

Automatic medical report generation (MRG) faces two key challenges: clinical reasoning inaccuracies leading to textual hallucinations (e.g., omissions or fabrications), and diagnostic bias stemming from insufficient rare-disease samples. To address these, we propose Chain-of-Medical-Thought (CoMT), the first framework to explicitly model physician diagnostic logic as a hierarchical, interpretable, fine-grained chain of thought. CoMT achieves clinical cognitive alignment in vision-language models via three innovations: radiological feature importance weighting, diagnosis-path-guided attention, and chain-of-thought supervised fine-tuning. Our method significantly improves robustness in rare-disease identification and report fidelity—reducing average hallucination rates by 38.2% and boosting key disease diagnostic accuracy by 12.7% across multiple public benchmarks. The code and curated dataset are publicly released.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Automatic medical report generation (MRG), which possesses significant research value as it can aid radiologists in clinical diagnosis and report composition, has garnered increasing attention. Despite recent progress, generating accurate reports remains arduous due to the requirement for precise clinical comprehension and disease diagnosis inference. Furthermore, owing to the limited accessibility of medical data and the imbalanced distribution of diseases, the underrepresentation of rare diseases in training data makes large-scale medical visual language models (LVLMs) prone to hallucinations, such as omissions or fabrications, severely undermining diagnostic performance and further intensifying the challenges for MRG in practice. In this study, to effectively mitigate hallucinations in medical report generation, we propose a chain-of-medical-thought approach (CoMT), which intends to imitate the cognitive process of human doctors by decomposing diagnostic procedures. The radiological features with different importance are structured into fine-grained medical thought chains to enhance the inferential ability during diagnosis, thereby alleviating hallucination problems and enhancing the diagnostic accuracy of MRG. The code and dataset have been released at https://github.com/FRENKIE-CHIANG/CoMT.

Problem

• Addresses hallucinations in medical report generation
• Improves diagnostic accuracy using structured thought chains
• Mitigates rare disease underrepresentation in training data

Innovation

• Chain-of-Medical-Thought approach (CoMT)
• Decomposes diagnostic procedures for accuracy
• Structures radiological features into thought chains