The Impact of Temporal Context Length and Encoding Strategies on Self-Supervised ECG Representation Learning

2026-08-12

AI Summary

This study addresses the limitations of existing self-supervised electrocardiogram (ECG) models, which often rely on short segments and discretized tokens, thereby discarding long-range temporal information critical for rhythm inference. The authors systematically evaluate the impact of varying temporal context lengths—from 16 seconds to 10 minutes—and front-end encoding strategies—continuous convolutional embeddings versus vector-quantized tokens—on representation learning, while keeping the Transformer backbone and training protocol fixed. Model performance is assessed through arrhythmia detection and patient-level retrieval tasks on the Icentia11k dataset. Results demonstrate that extended context significantly enhances transferability and cross-session stability, with models using 5–10 minutes of data achieving optimal performance. Moreover, continuous embeddings consistently outperform discrete tokens across all scales, yielding substantial gains in rhythm detection accuracy and retrieval precision, thereby offering crucial insights for designing clinically oriented ECG foundation models.

Abstract

Self-supervised electrocardiogram (ECG) models are often trained on a few seconds of ECG signal and, increasingly, on discretized token sequences. It remains unclear whether these choices sacrifice information needed for rhythm inference and longitudinal consistency in real-world ambulatory recordings. We present a controlled study on the Icentia11k single-lead dataset that varies (i) the input horizon (16 seconds, 1 minute, 5 minutes, and 10 minutes) and (ii) the front-end representation (continuous convolutional patch embeddings vs. fixed vector-quantized tokens), while holding the Transformer backbone and training protocol constant. Representations are assessed by downstream abnormal rhythm detection and by patient-level retrieval that probes cross-session stability. Our results show that increasing temporal context beyond 16-second snapshots yields stronger transfer and higher retrieval accuracy, with the strongest performance achieved by the 5- and 10-minute models, indicating improved capture of slow-varying rhythm dynamics and individual-specific structure. Across all evaluated horizons, continuous patch embeddings outperform discretized tokens, suggesting that quantization can discard clinically relevant waveform detail. These findings motivate ECG foundation models that emphasize extended context and continuous encoders for clinical prediction and similarity-based applications. Our code and pretrained models are publicly available at https://github.com/muha-0/ecg-ssl-representation-learning.

Problem

• temporal context
• self-supervised learning
• ECG representation
• rhythm inference
• longitudinal consistency

Innovation

• temporal context length
• continuous patch embeddings
• self-supervised ECG representation
• vector-quantized tokens
• longitudinal consistency