On the role of the tokenizer in ECG transformer models

2026-09-14

AI Summary

研究对比了八种ECG信号的分词策略，通过调整Transformer模型输入，发现与心电图形态对齐的分词方法能提高预测性能和内存效率。

Abstract

Tokenization determines both the physiological content presented to an ECG Transformer and the sequence over which attention operates. We compare eight tokenization strategies across Transformer, Informer, Reformer, and FEDformer on the nine-label CPSC2018 classification task. The input projection and principal backbone capacity are controlled to isolate the effect of token construction. Median-beat and HeartLang tokenization achieve mean macro-AUCs of 0.893 and 0.889 across the four backbones, compared with 0.822 and 0.824 for point-wise and patch-wise tokenization. Pooling the two physiology-aware representations yields an 8.2% relative improvement in macro-AUC. They also reduce mean sequence length from 1,250 to 158 tokens and mean peak training memory from 5.21 to 0.27 GB. The results show that aligning tokens with ECG morphology can improve both predictive performance and memory efficiency without increasing backbone capacity. The source code is available on https://github.com/LeeJarvis996/ecg_tokenizer.

Problem

• tokenizer
• ECG Transformer
• classification performance
• memory efficiency

Innovation

• tokenization strategies
• ECG morphology
• memory efficiency
• macro-AUC improvement