HeartBERT: A Self-Supervised ECG Embedding Model for Efficient and Effective Medical Signal Analysis

2024-11-08

arXiv.org

AI Summary

To address the challenges of scarce labeled data, high computational cost, and poor model generalization in ECG analysis, this paper proposes HeartBERT—the first self-supervised pretraining framework specifically designed for physiological time-series signals. Methodologically, it adapts the RoBERTa architecture for ECG modeling by introducing ECG-specific segmented normalization and a masked signal reconstruction pretraining objective, while incorporating a bidirectional LSTM head for downstream tasks. Experimentally, HeartBERT achieves superior performance on sleep staging and heartbeat classification: using only 30% of labeled data, it surpasses fully supervised state-of-the-art methods. It reduces parameter count by 42% and accelerates inference by 3.1×, significantly enhancing few-shot generalization capability.

Technology Category

• Machine Learning: Unsupervised & Self-Supervised Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

The HeartBert model is introduced with three primary objectives: reducing the need for labeled data, minimizing computational resources, and simultaneously improving performance in machine learning systems that analyze Electrocardiogram (ECG) signals. Inspired by Bidirectional Encoder Representations from Transformers (BERT) in natural language processing and enhanced with a self-supervised learning approach, the HeartBert model-built on the RoBERTa architecture-generates sophisticated embeddings tailored for ECG-based projects in the medical domain. To demonstrate the versatility, generalizability, and efficiency of the proposed model, two key downstream tasks have been selected: sleep stage detection and heartbeat classification. HeartBERT-based systems, utilizing bidirectional LSTM heads, are designed to address complex challenges. A series of practical experiments have been conducted to demonstrate the superiority and advancements of HeartBERT, particularly in terms of its ability to perform well with smaller training datasets, reduced learning parameters, and effective performance compared to rival models. The code and data are publicly available at https://github.com/ecgResearch/HeartBert.

Problem

• Reducing labeled data need for ECG analysis
• Minimizing computational resources in medical ML
• Improving ECG signal classification performance

Innovation

• Self-supervised ECG embedding model
• Based on RoBERTa architecture
• Utilizes bidirectional LSTM heads