EVL-ECG: Efficient ECG Interpretation With Multi-Aspect Heterogeneous Knowledge Distillation

2026-05-28

AI Summary

This work addresses the challenge of deploying high-fidelity electrocardiogram (ECG) interpretation models on clinical edge devices with limited computational resources, where conventional knowledge distillation struggles to effectively transfer the complex spatiotemporal dependencies and diagnostic reasoning of ECGs across heterogeneous architectures. To overcome this limitation, the authors propose EVL-ECG, a novel framework incorporating three ECG-aware innovations: multi-head cross-attention alignment, optimal transport–based visual feature matching, and geometric intra-architecture relation modeling, enabling efficient cross-architecture knowledge distillation. Evaluated on multiple ECG benchmarks, the method achieves performance gains of up to 2.4% in AUC and 1.1% in clinical accuracy, while yielding a lightweight ECG foundation model with only 2 billion parameters suitable for edge deployment.

Abstract

High-fidelity ECG interpretation is increasingly reliant on massive foundation models, yet their deployment in clinical edge-care remains hindered by extreme computational demands. While knowledge distillation (KD) is a promising solution, traditional methods fail to capture the complex spatio-temporal dependencies of ECG signals when transferring knowledge across heterogeneous architectures. In this paper, we propose EVL-ECG, a framework specifically designed for cross-architecture distillation of cardiac diagnostic logic. EVL-ECG introduces three ECG-aware innovations: (1) Multi-Head Cross-Attention Alignment, which harmonizes architectural discrepancies to preserve fine-grained morphological features; (2) Optimal Transport-based Visual Feature Matching, utilizing optimal transport to maintain global structural relationships across ECG leads despite mismatched token representations; and (3) Geometric Intra-Architecture Relation Matching, which distills the latent diagnostic reasoning of the teacher model. Evaluations across ECG benchmarks demonstrate that EVL-ECG yields improvements of up to 2.4% AUC and 1.1% clinical accuracy over existing baselines. Notably, EVL-ECG establishes an efficient 2B-parameter ECG foundation model, suitable for resource-constrained clinical environments.

Problem

• ECG interpretation
• knowledge distillation
• heterogeneous architectures
• spatio-temporal dependencies
• clinical edge deployment

Innovation

• Knowledge Distillation
• ECG Interpretation
• Optimal Transport
• Cross-Attention Alignment
• Heterogeneous Architectures