MSA-CNN: A Lightweight Multi-Scale CNN with Attention for Sleep Stage Classification

2025-01-06

AI Summary

To address the high parameter count and deployment challenges of existing sleep staging models, this paper proposes a lightweight multi-scale attentional convolutional network with only approximately 10,000 parameters. Methodologically, we introduce a novel complementary pooling multi-scale module that decouples spatiotemporal feature extraction, incorporate a low-cost global spatial convolution, and design a streamlined architecture aligned with clinical manual scoring logic. Evaluated on three public benchmark datasets, the proposed model consistently outperforms nine state-of-the-art methods, achieving superior accuracy and Cohen’s kappa—two primary evaluation metrics—while reducing parameter count by over 90%. This substantial compression significantly enhances computational efficiency and enables practical deployment on resource-constrained embedded platforms.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Multi-modal Vision

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Recent advancements in machine learning-based signal analysis, coupled with open data initiatives, have fuelled efforts in automatic sleep stage classification. Despite the proliferation of classification models, few have prioritised reducing model complexity, which is a crucial factor for practical applications. In this work, we introduce Multi-Scale and Attention Convolutional Neural Network (MSA-CNN), a lightweight architecture featuring as few as ~10,000 parameters. MSA-CNN leverages a novel multi-scale module employing complementary pooling to eliminate redundant filter parameters and dense convolutions. Model complexity is further reduced by separating temporal and spatial feature extraction and using cost-effective global spatial convolutions. This separation of tasks not only reduces model complexity but also mirrors the approach used by human experts in sleep stage scoring. We evaluated both small and large configurations of MSA-CNN against nine state-of-the-art baseline models across three public datasets, treating univariate and multivariate models separately. Our evaluation, based on repeated cross-validation and re-evaluation of all baseline models, demonstrated that the large MSA-CNN outperformed all baseline models on all three datasets in terms of accuracy and Cohen's kappa, despite its significantly reduced parameter count. Lastly, we explored various model variants and conducted an in-depth analysis of the key modules and techniques, providing deeper insights into the underlying mechanisms. The code for our models, baselines, and evaluation procedures is available at https://github.com/sgoerttler/MSA-CNN.

Problem

• Lightweight Deep Learning Model
• Automatic Sleep Stage Identification
• Computational Efficiency

Innovation

• MSA-CNN
• lightweight multi-scale attention
• sleep stage recognition