RadFabric: Agentic AI System with Reasoning Capability for Radiology

2025-06-17

AI Summary

Current CXR automated diagnosis systems suffer from narrow pathology coverage, low diagnostic accuracy, and fragmented multimodal reasoning. This paper introduces the first modular multi-agent multimodal reasoning system for CXR analysis built upon the Model Context Protocol (MCP), integrating visual recognition, anatomical localization, and clinical textual reasoning to enable fully automated, interpretable, and traceable radiological interpretation. We innovatively design an anatomical mapping agent and a preference-driven cross-modal alignment mechanism to support evidence-based diagnostic generation. On fracture detection, our system achieves perfect accuracy (1.000), while attaining an overall diagnostic accuracy of 0.799—substantially outperforming state-of-the-art methods (0.229–0.527). The system generates structured radiology reports that are anatomically precise, clinically verifiable, and rigorously supported by multimodal evidence.

Technology Category

• Computer Vision: Multi-modal Vision
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Chest X ray (CXR) imaging remains a critical diagnostic tool for thoracic conditions, but current automated systems face limitations in pathology coverage, diagnostic accuracy, and integration of visual and textual reasoning. To address these gaps, we propose RadFabric, a multi agent, multimodal reasoning framework that unifies visual and textual analysis for comprehensive CXR interpretation. RadFabric is built on the Model Context Protocol (MCP), enabling modularity, interoperability, and scalability for seamless integration of new diagnostic agents. The system employs specialized CXR agents for pathology detection, an Anatomical Interpretation Agent to map visual findings to precise anatomical structures, and a Reasoning Agent powered by large multimodal reasoning models to synthesize visual, anatomical, and clinical data into transparent and evidence based diagnoses. RadFabric achieves significant performance improvements, with near-perfect detection of challenging pathologies like fractures (1.000 accuracy) and superior overall diagnostic accuracy (0.799) compared to traditional systems (0.229 to 0.527). By integrating cross modal feature alignment and preference-driven reasoning, RadFabric advances AI-driven radiology toward transparent, anatomically precise, and clinically actionable CXR analysis.

Problem

• Improves pathology coverage and diagnostic accuracy in CXR analysis
• Integrates visual and textual reasoning for comprehensive CXR interpretation
• Enables modular and scalable AI system for radiology diagnostics

Innovation

• Multi-agent multimodal reasoning framework
• Model Context Protocol for modularity
• Cross-modal feature alignment integration