CT-GRAPH: Hierarchical Graph Attention Network for Anatomy-Guided CT Report Generation

2025-08-07

AI Summary

Existing CT report generation methods predominantly rely on global image features, neglecting fine-grained anatomical relationships among organs—thereby limiting diagnostic accuracy. To address this, we propose an anatomy-guided hierarchical graph modeling framework: (1) constructing a three-level graph structure—organ, organ system, and patient—based on anatomical masks; (2) designing a hierarchical graph attention network that explicitly fuses organ-level and global features extracted by a 3D pretrained encoder; and (3) integrating a large language model for structured report generation. This work is the first to enable interpretable, fine-grained anatomical knowledge modeling in CT report generation. Evaluated on the CT-RATE dataset, our method achieves an absolute F1-score improvement of 7.9% over state-of-the-art approaches. The implementation is publicly available.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Reasoning under Uncertainty: Graphical Models

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

As medical imaging is central to diagnostic processes, automating the generation of radiology reports has become increasingly relevant to assist radiologists with their heavy workloads. Most current methods rely solely on global image features, failing to capture fine-grained organ relationships crucial for accurate reporting. To this end, we propose CT-GRAPH, a hierarchical graph attention network that explicitly models radiological knowledge by structuring anatomical regions into a graph, linking fine-grained organ features to coarser anatomical systems and a global patient context. Our method leverages pretrained 3D medical feature encoders to obtain global and organ-level features by utilizing anatomical masks. These features are further refined within the graph and then integrated into a large language model to generate detailed medical reports. We evaluate our approach for the task of report generation on the large-scale chest CT dataset CT-RATE. We provide an in-depth analysis of pretrained feature encoders for CT report generation and show that our method achieves a substantial improvement of absolute 7.9% in F1 score over current state-of-the-art methods. The code is publicly available at https://github.com/hakal104/CT-GRAPH.

Problem

• Automate radiology report generation to assist radiologists
• Model fine-grained organ relationships for accurate reporting
• Integrate anatomical knowledge with global patient context

Innovation

• Hierarchical graph attention network for CT reports
• Pretrained 3D medical feature encoders integration
• Anatomical graph linking organ to system features