MRGAgents: A Multi-Agent Framework for Improved Medical Report Generation with Med-LVLMs

2025-05-24

AI Summary

Med-LVLMs suffer from two critical limitations in radiology report generation: (1) excessive prediction of “normal” findings, leading to missed key abnormalities, and (2) insufficient systematic anatomical region coverage. To address these, we propose MRGAgents—a novel multi-agent framework that introduces disease-category-specific specialized agents, coordinated via a hierarchical collaboration mechanism. Each agent is fine-tuned on disease-specific subsets derived from IU X-ray and MIMIC-CXR, enabling fine-grained, pathology-aware adaptation. The framework integrates multi-agent collaborative reasoning, Med-LVLM customization, and structured report generation. Experimental results demonstrate significant improvements over state-of-the-art methods across abnormality detection rate, anatomical region coverage, and clinical completeness—substantially enhancing diagnostic utility and report reliability.

Technology Category

• Multiagent Systems: Modeling other Agents
• Computer Vision: Multi-modal Vision
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Medical Large Vision-Language Models (Med-LVLMs) have been widely adopted for medical report generation. Despite Med-LVLMs producing state-of-the-art performance, they exhibit a bias toward predicting all findings as normal, leading to reports that overlook critical abnormalities. Furthermore, these models often fail to provide comprehensive descriptions of radiologically relevant regions necessary for accurate diagnosis. To address these challenges, we proposeMedical Report Generation Agents (MRGAgents), a novel multi-agent framework that fine-tunes specialized agents for different disease categories. By curating subsets of the IU X-ray and MIMIC-CXR datasets to train disease-specific agents, MRGAgents generates reports that more effectively balance normal and abnormal findings while ensuring a comprehensive description of clinically relevant regions. Our experiments demonstrate that MRGAgents outperformed the state-of-the-art, improving both report comprehensiveness and diagnostic utility.

Problem

• Med-LVLMs bias toward normal findings, missing critical abnormalities
• Models lack comprehensive descriptions of key diagnostic regions
• Need for balanced, disease-specific medical report generation

Innovation

• Multi-agent framework for medical report generation
• Fine-tunes disease-specific agents using curated datasets
• Improves report comprehensiveness and diagnostic utility