RADAR: Enhancing Radiology Report Generation with Supplementary Knowledge Injection

2025-05-20

AI Summary

Existing radiology report generation methods typically superimpose externally retrieved knowledge onto the implicit medical knowledge embedded in large language models (LLMs), leading to information redundancy and inefficient representation utilization. To address this, we propose a collaborative enhancement framework that, for the first time, systematically decouples and synergistically integrates the LLM’s implicit endogenous knowledge with external retrieval knowledge. Specifically, expert-guided image–text alignment is employed to extract endogenous knowledge, while a dual-source knowledge aggregation mechanism ensures complementary—rather than redundant—integration of both knowledge sources. Implemented atop a multimodal LLM, our method achieves state-of-the-art performance across MIMIC-CXR, CheXpert-Plus, and IU X-ray benchmarks. It significantly improves report fluency, clinical accuracy, and diagnostic consistency, demonstrating superior language quality and clinical reliability compared to prior approaches.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities in various domains, including radiology report generation. Previous approaches have attempted to utilize multimodal LLMs for this task, enhancing their performance through the integration of domain-specific knowledge retrieval. However, these approaches often overlook the knowledge already embedded within the LLMs, leading to redundant information integration and inefficient utilization of learned representations. To address this limitation, we propose RADAR, a framework for enhancing radiology report generation with supplementary knowledge injection. RADAR improves report generation by systematically leveraging both the internal knowledge of an LLM and externally retrieved information. Specifically, it first extracts the model's acquired knowledge that aligns with expert image-based classification outputs. It then retrieves relevant supplementary knowledge to further enrich this information. Finally, by aggregating both sources, RADAR generates more accurate and informative radiology reports. Extensive experiments on MIMIC-CXR, CheXpert-Plus, and IU X-ray demonstrate that our model outperforms state-of-the-art LLMs in both language quality and clinical accuracy

Problem

• Enhancing radiology report generation with knowledge injection
• Leveraging internal and external knowledge for accurate reports
• Improving language quality and clinical accuracy in radiology reports

Innovation

• Leverages internal LLM knowledge and external retrieval
• Extracts model knowledge matching expert classifications
• Aggregates internal and external knowledge for reports