Echo2ECG: Enhancing ECG Representations with Cardiac Morphology from Multi-View Echos

2026-03-09

AI Summary

This study addresses the limitation of electrocardiography (ECG) in capturing cardiac morphological characteristics—such as left ventricular ejection fraction—which hinders its utility in structural heart disease screening. To overcome this, the authors propose a multimodal self-supervised learning framework that aligns ECG signals with multi-view echocardiograms (Echo), thereby integrating more comprehensive structural information and mitigating representation mismatches caused by single-view Echo. The approach yields a compact ECG feature extractor that outperforms existing methods on both structural heart disease classification and ECG-based Echo retrieval across three datasets, while achieving a model size merely 1/18th that of the largest baseline.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Electrocardiography (ECG) is a low-cost, widely used modality for diagnosing electrical abnormalities like atrial fibrillation by capturing the heart's electrical activity. However, it cannot directly measure cardiac morphological phenotypes, such as left ventricular ejection fraction (LVEF), which typically require echocardiography (Echo). Predicting these phenotypes from ECG would enable early, accessible health screening. Existing self-supervised methods suffer from a representational mismatch by aligning ECGs to single-view Echos, which only capture local, spatially restricted anatomical snapshots. To address this, we propose Echo2ECG, a multimodal self-supervised learning framework that enriches ECG representations with the heart's morphological structure captured in multi-view Echos. We evaluate Echo2ECG as an ECG feature extractor on two clinically relevant tasks that fundamentally require morphological information: (1) classification of structural cardiac phenotypes across three datasets, and (2) retrieval of Echo studies with similar morphological characteristics using ECG queries. Our extracted ECG representations consistently outperform those of state-of-the-art unimodal and multimodal baselines across both tasks, despite being 18x smaller than the largest baseline. These results demonstrate that Echo2ECG is a robust, powerful ECG feature extractor. Our code is accessible at https://github.com/michelleespranita/Echo2ECG.

Problem

• ECG
• cardiac morphology
• echocardiography
• left ventricular ejection fraction
• multimodal representation

Innovation

• multimodal self-supervised learning
• multi-view echocardiography
• ECG representation learning
• cardiac morphology
• cross-modal alignment