EchoSonar-R: A Multi-View Reasoning-Enabled Model for Disease Classification and Report Generation in Echocardiography

2026-06-26

AI Summary

This study addresses the limited clinical trust in existing echocardiography analysis models, which often lack explicit diagnostic reasoning and anatomical spatial evidence. To overcome this, the authors propose a multi-view reasoning-driven vision-language model that jointly performs multi-label disease classification and structured report generation. The approach integrates a structure-aware cardiac detector to provide anatomical localization cues and combines a spatiotemporal video encoder with a two-stage training strategy—supervised fine-tuning followed by Group Relative Policy Optimization—to unify optimization for both classification and reporting. Experiments demonstrate that the model improves macro-averaged balanced accuracy by 17.1% on a private dataset and by 6.1% on MIMICEchoQA, achieves a clinical credibility GREEN score of 0.800, and produces interpretable diagnostic pathways grounded in multi-view visual evidence.

Abstract

Echocardiography is the most widely used non-invasive cardiac imaging modality, providing essential information for cardiovascular diagnosis. Interpreting an echocardiogram requires synthesizing complementary evidence across multiple heart views to identify abnormalities and produce structured clinical reports. While recent efforts focus on improving classification performance, most models lack explicit diagnostic reasoning and spatially grounded anatomical evidence, limiting clinician trust. We present EchoSonar-R, a multi-view reasoning-enabled vision-language model that jointly performs multi-label disease classification and report generation from echocardiography studies. EchoSonar-R combines a spatiotemporal video encoder with a structure-aware cardiac detector that provides spatially grounded anatomical cues to improve interpretability and clinician trust during cross-view reasoning. EchoSonar-R is trained in two stages: supervised fine-tuning (SFT) on reasoning-annotated targets, followed by Group Relative Policy Optimization (GRPO) with task-specific rewards that jointly align classification and report generation within a unified reinforcement-learning framework. Across a private multi-view dataset and two public benchmarks, EchoSonar-R improves macro balanced accuracy by 17.1% on the private set and 6.1% on MIMICEchoQA over the strongest baseline, achieves a GREEN clinical faithfulness score of 0.800, and produces interpretable reasoning traces grounded in multi-view visual evidence.

Problem

• echocardiography
• disease classification
• report generation
• diagnostic reasoning
• anatomical evidence

Innovation

• multi-view reasoning
• vision-language model
• spatially grounded anatomy
• reinforcement learning
• echocardiography report generation