Translating Electrocardiograms to Cardiac Magnetic Resonance Imaging Useful for Cardiac Assessment and Disease Screening: A Multi-Center Study AI for ECG to CMR Translation Study

2024-11-19

AI Summary

Cardiovascular disease diagnosis urgently requires tools that balance accessibility and diagnostic precision: cardiac magnetic resonance (CMR) is the gold-standard imaging modality but remains costly and inaccessible; electrocardiography (ECG) is widely available and low-cost yet lacks quantitative structural and functional assessment capability. This work proposes the first cross-modal pretraining framework integrating contrastive learning and masked autoregressive generation to enable end-to-end synthesis of high-fidelity CMR images and quantitative functional parameters—including ejection fraction and ventricular volumes—from 12-lead ECG alone. Evaluated on multicenter real-world clinical data under a federated learning paradigm, our method achieves a 24.8% improvement in phenotypic regression R², a 39.3% gain in cardiomyopathy classification AUC, and a 36.6% increase in generated image SSIM on UK Biobank. Notably, ECG-only diagnosis accuracy surpasses that of clinicians using both ECG and ground-truth CMR by 13.9%, demonstrating for the first time the feasibility of high-precision cardiac imaging reconstruction and disease screening from unimodal physiological signals.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Cardiovascular diseases (CVDs) are the leading cause of global mortality, necessitating accessible and accurate diagnostic tools. While cardiac magnetic resonance imaging (CMR) provides gold-standard insights into cardiac structure and function, its clinical utility is limited by high cost and complexity. In contrast, electrocardiography (ECG) is inexpensive and widely available but lacks the granularity of CMR. We propose CardioNets, a deep learning framework that translates 12-lead ECG signals into CMR-level functional parameters and synthetic images, enabling scalable cardiac assessment. CardioNets integrates cross-modal contrastive learning and generative pretraining, aligning ECG with CMR-derived cardiac phenotypes and synthesizing high-resolution CMR images via a masked autoregressive model. Trained on 159,819 samples from five cohorts, including the UK Biobank (n=42,483) and MIMIC-IV-ECG (n=164,550), and externally validated on independent clinical datasets (n=3,767), CardioNets achieved strong performance across disease screening and phenotype estimation tasks. In the UK Biobank, it improved cardiac phenotype regression R2 by 24.8% and cardiomyopathy AUC by up to 39.3% over baseline models. In MIMIC, it increased AUC for pulmonary hypertension detection by 5.6%. Generated CMR images showed 36.6% higher SSIM and 8.7% higher PSNR than prior approaches. In a reader study, ECG-only CardioNets achieved 13.9% higher accuracy than human physicians using both ECG and real CMR. These results suggest that CardioNets offers a promising, low-cost alternative to CMR for large-scale CVD screening, particularly in resource-limited settings. Future efforts will focus on clinical deployment and regulatory validation of ECG-based synthetic imaging.

Problem

• Translating ECG signals into CMR-level cardiac parameters for affordable diagnosis
• Overcoming CMR's high cost and complexity using deep learning on ECG data
• Enhancing CVD screening accuracy with AI-generated synthetic CMR images

Innovation

• Deep learning translates ECG to CMR images
• Cross-modal contrastive learning aligns ECG with CMR
• Generative pretraining synthesizes high-resolution CMR images