CogRad: A Cognitively-Inspired Multi-Agent Framework for Radiology Report Generation

2026-07-04

AI Summary

This work addresses the limitations of existing radiology report generation methods, which typically rely on single-pass forward generation and lack mechanisms for verifying image evidence or revising previously generated content. The authors propose the first cognitively inspired, four-agent framework: Scout identifies suspicious regions, Investigator performs focused analysis, Writer generates linguistic prefixes, and Verifier employs a visual entailment loss to supervise training and iteratively validate and refine each sentence during inference. By integrating dynamic visual verification and iterative correction, the approach significantly enhances clinical credibility. Leveraging Slot Attention, disease-gated visual prefixes, large language models, and Grad-CAM visualizations, the method achieves state-of-the-art BLEU-4 and CIDEr scores on CheXpert Plus and IU X-Ray datasets, with further validation through RadGraph F1, CheXbert F1, and hallucination analyses confirming superior clinical accuracy.

Abstract

Automated radiology report generation (RRG) can ease radiologist workload, yet most existing systems produce a report in a single forward pass, with no mechanism to check a claim against the image or revisit a finding once stated. We present CogRad, a cognitively inspired multi-agent framework that structures generation around four stages of a radiologist's reading process. A Scout agent discovers anatomical regions directly from image patches via slot attention and assigns region and disease-level triage scores; an Investigator agent concentrates representational capacity on the regions Scout flags as suspicious; a Writer agent compiles these signals into a disease gated visual prefix for a large language model; and a Verifier agent supervises training with a visual entailment loss and, at inference, re-examines its own draft sentence by sentence, regenerating any report it judges insufficiently grounded. On CheXpert Plus, CogRad attains a BLEU-4 of 0.316 and a CIDEr of 0.322, the best scores among the methods we compare against. On IU X-Ray, it attains a BLEU-4 of 0.201 and a CIDEr of 0.724, leading every baseline on every standard NLG metric. We further evaluate CogRad with RadGraph F1, CheXbert F1, and a hallucination analysis to assess clinical accuracy beyond standard text-overlap metrics, complemented by ablation studies and Grad-CAM-based visualizations that characterize each agent's contribution and the model's visual grounding.

Problem

• radiology report generation
• visual grounding
• hallucination
• multi-agent framework
• automated diagnosis

Innovation

• multi-agent framework
• cognitive inspiration
• visual grounding
• radiology report generation
• iterative verification