Beyond Independent Frames: Latent Attention Masked Autoencoders for Multi-View Echocardiography

2026-04-16

AI Summary

Traditional masked autoencoders struggle to model the multi-view, sparse, and heterogeneous spatiotemporal structure inherent in echocardiography, resulting in representations that lack holistic coherence and consistency. This work proposes the Latent-space Attention Masked Autoencoder (LAMAE), which, for the first time, integrates multi-view structural priors into a foundational model by introducing cross-frame and cross-view attention mechanisms in the latent space, enabling unified modeling of variable-length sequences and heterogeneous views. Pretraining on the large-scale clinical MIMIC-IV-ECHO dataset demonstrates that LAMAE effectively reconstructs comprehensive cardiac functional representations from partial observations and accurately predicts ICD-10 diagnostic codes. Moreover, the learned representations exhibit strong transferability and robustness when applied across age groups, notably from adult to pediatric populations.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

Echocardiography is a widely used modality for cardiac assessment due to its non-invasive and cost-effective nature, but the sparse and heterogeneous spatiotemporal views of the heart pose distinct challenges. Existing masked autoencoder (MAE) approaches typically process images or short clips independently, failing to capture the inherent multi-view structure required for coherent cardiac representation. We introduce Latent Attention Masked Autoencoder (LAMAE), a foundation model architecture tailored to the multi-view nature of medical imaging. LAMAE augments the standard MAE with a latent attention module that enables information exchange across frames and views directly in latent space. This allows the model to aggregate variable-length sequences and distinct views, reconstructing a holistic representation of cardiac function from partial observations. We pretrain LAMAE on MIMIC-IV-ECHO, a large-scale, uncurated dataset reflecting real-world clinical variability. To the best of our knowledge, we present the first results for predicting ICD-10 codes from MIMIC-IV-ECHO videos. Furthermore, we empirically demonstrate that representations learned from adult data transfer effectively to pediatric cohorts despite substantial anatomical differences. These results provide evidence that incorporating structural priors, such as multi-view attention, yields significantly more robust and transferable representations.

Problem

• multi-view echocardiography
• masked autoencoder
• spatiotemporal representation
• cardiac assessment
• heterogeneous views

Innovation

• Latent Attention
• Masked Autoencoder
• Multi-View Echocardiography
• Foundation Model
• Cross-View Representation