Global and Local Contrastive Learning for Joint Representations from Cardiac MRI and ECG

2025-06-24

AI Summary

To address the clinical dilemma wherein electrocardiography (ECG) lacks direct functional assessment capability (e.g., ventricular volume, ejection fraction) while cardiac magnetic resonance (CMR) remains costly and inaccessible, this work proposes PTACL—a self-supervised multimodal contrastive learning framework. PTACL jointly leverages spatiotemporal information from paired ECG and CMR signals to achieve patient-level global alignment and beat-level local temporal alignment—without introducing additional learnable parameters. Trained on 27,951 matched ECG–CMR samples from UK Biobank, PTACL significantly improves ECG-based prediction accuracy for cardiac functional parameters and enables cross-modal patient phenotypic retrieval. Its core innovation lies in a dual-granularity contrastive mechanism that drives fine-grained, cross-modal representation alignment—marking the first demonstration that ECG-derived representations can approach CMR-level functional assessment capability.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

An electrocardiogram (ECG) is a widely used, cost-effective tool for detecting electrical abnormalities in the heart. However, it cannot directly measure functional parameters, such as ventricular volumes and ejection fraction, which are crucial for assessing cardiac function. Cardiac magnetic resonance (CMR) is the gold standard for these measurements, providing detailed structural and functional insights, but is expensive and less accessible. To bridge this gap, we propose PTACL (Patient and Temporal Alignment Contrastive Learning), a multimodal contrastive learning framework that enhances ECG representations by integrating spatio-temporal information from CMR. PTACL uses global patient-level contrastive loss and local temporal-level contrastive loss. The global loss aligns patient-level representations by pulling ECG and CMR embeddings from the same patient closer together, while pushing apart embeddings from different patients. Local loss enforces fine-grained temporal alignment within each patient by contrasting encoded ECG segments with corresponding encoded CMR frames. This approach enriches ECG representations with diagnostic information beyond electrical activity and transfers more insights between modalities than global alignment alone, all without introducing new learnable weights. We evaluate PTACL on paired ECG-CMR data from 27,951 subjects in the UK Biobank. Compared to baseline approaches, PTACL achieves better performance in two clinically relevant tasks: (1) retrieving patients with similar cardiac phenotypes and (2) predicting CMR-derived cardiac function parameters, such as ventricular volumes and ejection fraction. Our results highlight the potential of PTACL to enhance non-invasive cardiac diagnostics using ECG. The code is available at: https://github.com/alsalivan/ecgcmr

Problem

• Bridging ECG and CMR data gaps for enhanced cardiac diagnostics
• Improving ECG representations with CMR-derived spatio-temporal insights
• Enabling non-invasive prediction of cardiac function parameters

Innovation

• Multimodal contrastive learning for ECG-CMR integration
• Global and local alignment losses for patient and temporal matching
• Enhanced ECG representations without additional learnable weights