MonteRET: AI Agent Enhancing Multimodal LLMs with Multi-granularity Knowledge Retrieval for Chest CT Report Generation

2026-07-15

AI Summary

Existing methods for automated chest CT report generation struggle to simultaneously capture global semantic context and fine-grained anatomical details, often leading to missed critical findings. This work proposes MonteRET, a novel framework that introduces, for the first time, a multi-granularity knowledge retrieval mechanism coupled with a region-aware rewriting agent. By leveraging a multimodal large language model, MonteRET integrates holistic CT features with region-level vision–language aligned representations to retrieve relevant medical knowledge, which then guides a knowledge-informed rewriting agent to refine the initial draft report. Evaluated on RadGenome-ChestCT and external test sets, the method significantly improves report quality, semantic similarity, and clinical validity—particularly enhancing recall to reduce missed diagnoses—and receives favorable assessment from radiology residents in human evaluations.

Abstract

Automated chest CT report generation remains challenging because clinically faithful reporting requires both whole-volume understanding and accurate description of localized anatomical findings. Here we developed and retrospectively evaluated MonteRET, a region-aware retrieval-enhanced framework for generating chest CT findings sections. MonteRET integrates global CT features with region-level anatomical representations, retrieves clinically relevant knowledge using predicted medical conditions and region-level vision-language alignment, and refines initial reports through a knowledge-guided report rewriting agent. We trained our model on a public cohort with 24,128 CT scans from RadGenome-ChestCT. We evaluated MonteRET on the public RadGenome-ChestCT test set of 1,564 CT scans and an external cohort of 82 CT scans from NewYork-Presbyterian/Weill Cornell Medical Center. MonteRET improved report quality, semantic similarity, and clinical efficacy compared with a matched baseline and several state-of-the-art methods. Gains were most pronounced for recall, suggesting fewer omitted findings. Human expert evaluation by radiology residents also favored MonteRET.

Problem

• chest CT report generation
• multimodal LLMs
• knowledge retrieval
• anatomical findings
• clinical reporting

Innovation

• retrieval-enhanced generation
• multimodal LLMs
• multi-granularity knowledge
• region-aware representation
• AI agent