BAAI Cardiac Agent: An intelligent multimodal agent for automated reasoning and diagnosis of cardiovascular diseases from cardiac magnetic resonance imaging

2026-04-05

AI Summary

Cardiac magnetic resonance (CMR) imaging interpretation is highly dependent on expert experience and inefficient due to its multi-sequence, multi-phase nature and complex quantitative analysis. This work proposes the first end-to-end multimodal agent system that automates the entire CMR workflow by dynamically coordinating multiple specialized expert models. The system integrates multimodal deep learning, parametric regression, classification, and natural language generation to perform cardiac structure segmentation, functional quantification, tissue characterization, disease diagnosis, and structured report generation. Validated on 2,413 patient cases, the system demonstrates strong agreement with clinical measurements for key metrics such as left ventricular ejection fraction (r > 0.90) and achieves internal and external diagnostic AUCs of 0.93 and 0.81, respectively, outperforming existing methods and producing reports comparable in quality to those of radiology experts.

Abstract

Cardiac magnetic resonance (CMR) is a cornerstone for diagnosing cardiovascular disease. However, it remains underutilized due to complex, time-consuming interpretation across multi-sequences, phases, quantitative measures that heavily reliant on specialized expertise. Here, we present BAAI Cardiac Agent, a multimodal intelligent system designed for end-to-end CMR interpretation. The agent integrates specialized cardiac expert models to perform automated segmentation of cardiac structures, functional quantification, tissue characterization and disease diagnosis, and generates structured clinical reports within a unified workflow. Evaluated on CMR datasets from two hospitals (2413 patients) spanning 7-types of major cardiovascular diseases, the agent achieved an area under the receiver-operating-characteristic curve exceeding 0.93 internally and 0.81 externally. In the task of estimating left ventricular function indices, the results generated by this system for core parameters such as ejection fraction, stroke volume, and left ventricular mass are highly consistent with clinical reports, with Pearson correlation coefficients all exceeding 0.90. The agent outperformed state-of-the-art models in segmentation and diagnostic tasks, and generated clinical reports showing high concordance with expert radiologists (six readers across three experience levels). By dynamically orchestrating expert models for coordinated multimodal analysis, this agent framework enables accurate, efficient CMR interpretation and highlights its potentials for complex clinical imaging workflows. Code is available at https://github.com/plantain-herb/Cardiac-Agent.

Problem

• cardiac magnetic resonance
• cardiovascular disease
• automated interpretation
• clinical diagnosis
• multimodal imaging

Innovation

• multimodal agent
• cardiac MRI interpretation
• automated diagnosis
• expert model orchestration
• structured clinical reporting