Automated Structured Radiology Report Generation with Rich Clinical Context

2025-09-30

AI Summary

Existing SRRG methods neglect clinical context, leading to temporal hallucinations—such as fabricated medical history—and undermining report reliability. To address this, we propose C-SRRG, the first framework that systematically integrates multi-view X-ray images, clinical indications, imaging parameters, and prior medical history into a unified multimodal clinical context. Leveraging a multimodal large language model, C-SRRG jointly models radiographic imagery and structured clinical text to ensure temporally consistent, clinically grounded reasoning. This design inherently suppresses temporal hallucinations at their source, significantly improving diagnostic logical coherence and report accuracy. On standard benchmarks, C-SRRG comprehensively outperforms state-of-the-art methods, yielding higher-quality, clinically aligned reports. To foster reproducibility and advancement in clinically aware automated radiology reporting, we publicly release our code, dataset, and model weights.

Technology Category

• Computer Vision: Multi-modal Vision
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Automated structured radiology report generation (SRRG) from chest X-ray images offers significant potential to reduce workload of radiologists by generating reports in structured formats that ensure clarity, consistency, and adherence to clinical reporting standards. While radiologists effectively utilize available clinical contexts in their diagnostic reasoning, existing SRRG systems overlook these essential elements. This fundamental gap leads to critical problems including temporal hallucinations when referencing non-existent clinical contexts. To address these limitations, we propose contextualized SRRG (C-SRRG) that comprehensively incorporates rich clinical context for SRRG. We curate C-SRRG dataset by integrating comprehensive clinical context encompassing 1) multi-view X-ray images, 2) clinical indication, 3) imaging techniques, and 4) prior studies with corresponding comparisons based on patient histories. Through extensive benchmarking with state-of-the-art multimodal large language models, we demonstrate that incorporating clinical context with the proposed C-SRRG significantly improves report generation quality. We publicly release dataset, code, and checkpoints to facilitate future research for clinically-aligned automated RRG at https://github.com/vuno/contextualized-srrg.

Problem

• Existing radiology report systems ignore essential clinical context elements
• Current methods cause temporal hallucinations by referencing non-existent clinical data
• Automated report generation lacks integration of rich patient history and imaging context

Innovation

• Incorporates rich clinical context for structured radiology reporting
• Integrates multi-view images and patient history data
• Uses multimodal large language models for improved generation