LSTrans: Efficient Knowledge Transfer for Lightweight and Automated ECG Classification

2026-07-12

AI Summary

This work addresses the challenge of high computational overhead in deploying deep learning models for automated electrocardiogram (ECG) classification on resource-constrained wearable devices. To this end, the authors propose LSTrans, a lightweight hybrid architecture that integrates a specialized one-dimensional convolutional backbone—designed with interleaved layers to simultaneously capture macroscopic rhythm and microscopic morphological features—with a Transformer encoder. Parameter efficiency is further enhanced through low-rank adaptation (LoRA). The approach also leverages both homogeneous and heterogeneous knowledge distillation strategies to effectively transfer diagnostic capabilities from a large-capacity teacher model. Experimental results demonstrate that LSTrans significantly reduces peak memory consumption and training latency across multiple benchmark datasets while achieving an excellent trade-off between diagnostic sensitivity and resource efficiency.

Abstract

Deploying deep learning models for automated electrocardiogram classification on resource-constrained wearable devices remains challenging due to high computational costs. To address this, we propose LSTrans, a lightweight hybrid model designed for efficient and sensitive ECG analysis. LSTrans introduces a specialized 1D convolutional backbone with an interleaved layer architecture to capture both macroscopic rhythmic trends and microscopic morphological variations. This backbone is cascaded with a Transformer encoder to model long-range temporal dependencies, incorporating Low-Rank Adaptation across critical layers to compress the model and reduce the trainable parameter space. We further employ homogeneous and heterogeneous knowledge distillation to transfer diagnostic expertise from high-capacity teacher models to the student. Experimental results on multiple benchmark datasets demonstrate that LSTrans achieves a competitive balance between diagnostic sensitivity and resource efficiency, substantially reducing peak memory footprints and training latency during downstream adaptation. The source code is available for review at https://github.com/zyee00128/LSTrans4BIBM.

Problem

• ECG classification
• resource-constrained devices
• computational cost
• wearable devices
• automated diagnosis

Innovation

• Lightweight Model
• Knowledge Distillation
• Low-Rank Adaptation
• ECG Classification
• Transformer