Spatio-Temporal Fusion Model for Standard View Classification of Echocardiographic Videos

2026-06-15

AI Summary

This study addresses the challenges of automatic classification of standard echocardiographic views, including scarce public data, heterogeneous frame quality, and difficulties in spatiotemporal modeling. To this end, the authors introduce and publicly release EV9V, the first large-scale video dataset comprising nine standard echocardiographic view classes, and conduct a systematic evaluation of mainstream video models, including CNNs, RNNs, and Transformers. They propose a dual-stream CNN-LSTM architecture that integrates uncertainty-aware learning with an evidence-theoretic fusion mechanism, selectively leveraging high-quality segments during training and dynamically aggregating multi-frame evidence during inference to enhance robustness against frame quality variations. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on EV9V, confirming its effectiveness and generalizability.

Abstract

Automated classification of standard echocardiographic views is crucial for efficient clinical workflow but faces three main challenges. First, publicly available datasets are scarce and limited in scale and view coverage. Second, the performance of some modern video-level architectures for echocardiographic view classification remains underexplored. Third, some view categories exhibit highly similar spatial appearances, making single-frame features insufficient for discrimination, while heterogeneous frame quality complicates robust temporal information fusion. To address these challenges, we release the Echocardiographic Videos of Nine Views (EV9V) dataset, comprising 5,138 videos, 910,579 frames, and 9 standard views, which is, to the best of our knowledge, the largest publicly available echocardiography video dataset. Using EV9V, we systematically benchmark representative video classification architectures, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Transformers. Furthermore, we propose a Spatio-Temporal Fusion Model (STFM), an efficient dual-stream CNN-LSTM (Long Short-Term Memory) framework that jointly captures spatial anatomical structures and temporal cardiac dynamics. The proposed framework leverages uncertainty-aware learning to preferentially sample representative video segments during training and evidence-based fusion during inference, improving robustness to variations in frame quality across echocardiographic videos. Extensive experiments demonstrate that our method achieves competitive performance across diverse video classification models, validating the effectiveness of uncertainty-aware spatio-temporal learning for echocardiographic view classification. The code is available at https://github.com/bgx666/stfm.

Problem

• echocardiographic view classification
• spatio-temporal fusion
• video-level architectures
• frame quality heterogeneity
• standard view discrimination

Innovation

• Spatio-Temporal Fusion
• Uncertainty-aware Learning
• Echocardiographic View Classification
• Dual-stream CNN-LSTM
• Video Benchmarking