Learning Fingerprints for Medical Time Series with Redundancy-Constrained Information Maximization

2026-04-30

AI Summary

This work addresses the challenge of learning compact and interpretable representations from high-dimensional, variable-length, and noisy medical time-series signals—such as ECG and EEG—by proposing a novel “disentangled rate–distortion” framework. For the first time in medical time-series modeling, the approach explicitly optimizes both information sufficiency and statistical disentanglement among latent tokens. It compresses input sequences into a fixed number of fingerprint tokens via a cross-attention bottleneck and jointly minimizes reconstruction loss with a diversity regularizer based on Total Coding Rate (TCR). The resulting low-dimensional representations are sample-efficient, minimally redundant, and structured such that each token corresponds to an independent factor of variation, thereby substantially enhancing robustness, interpretability, and their potential utility as digital biomarkers.

Abstract

Learning meaningful representations from medical time series (MedTS) such as ECG or EEG signals is a critical challenge. These signals are often high-dimensional, variable-length and rife with noise. Existing self-supervised approaches, such as Masked Autoencoders (MAEs) are highly effective for pre-training general-purpose encoders. However, they do not explicitly learn compact and semantically interpretable latent representations, typically relying on heuristic aggregation strategies such as global average pooling or a designated [CLS] token. We propose a novel framework that compresses a variable-length MedTS into a fixed-size set of $k$ latent Fingerprint Tokens. Our architecture employs a cross-attention bottleneck to generate these tokens and is trained with a dual-objective function. The first objective is a reconstruction loss, which ensures the tokens are \textit{sufficient statistics} for the original data. The second, a diversity penalty based on the Total Coding Rate (TCR), explicitly minimizes the redundancy between tokens, encouraging them to become statistically \textit{disentangled} representations. We present the theoretical justification for our method, framing it as a novel \textbf{Disentangled Rate-Distortion} problem. This approach produces a low-dimensional, interpretable, and sample-efficient representation, where each token is encouraged to capture an independent factor of variation, paving the way for more robust digital biomarkers.

Problem

• medical time series
• compact representation
• semantic interpretability
• redundancy minimization
• disentangled representation

Innovation

• Fingerprint Tokens
• Disentangled Representation
• Total Coding Rate
• Cross-Attention Bottleneck
• Rate-Distortion