SPECTRA: An Efficient Spectral-Informed Neural Network for Sensor-Based Activity Recognition

2026-03-27

AI Summary

This work proposes a deployment-oriented, lightweight spectral-aware neural network to address the challenges of real-time, low-power, and privacy-preserving human activity recognition (HAR) on edge devices. The architecture uniquely integrates short-time Fourier transform (STFT) explicitly into an end-to-end HAR model, combining depthwise separable convolutions, channel-wise self-attention, compact bidirectional GRUs, and attention-based pooling to efficiently capture time-frequency dependencies. Evaluated on five public datasets, the model achieves performance comparable to large CNN, LSTM, and Transformer baselines while significantly reducing parameter count, inference latency, and energy consumption. The proposed approach has been successfully deployed on both a Google Pixel 9 smartphone and an STM32L4 microcontroller, demonstrating its practicality for resource-constrained edge environments.

Abstract

Real time sensor based applications in pervasive computing require edge deployable models to ensure low latency privacy and efficient interaction. A prime example is sensor based human activity recognition where models must balance accuracy with stringent resource constraints. Yet many deep learning approaches treat temporal sensor signals as black box sequences overlooking spectral temporal structure while demanding excessive computation. We present SPECTRA a deployment first co designed spectral temporal architecture that integrates short time Fourier transform STFT feature extraction depthwise separable convolutions and channel wise self attention to capture spectral temporal dependencies under real edge runtime and memory constraints. A compact bidirectional GRU with attention pooling summarizes within window dynamics at low cost reducing downstream model burden while preserving accuracy. Across five public HAR datasets SPECTRA matches or approaches larger CNN LSTM and Transformer baselines while substantially reducing parameters latency and energy. Deployments on a Google Pixel 9 smartphone and an STM32L4 microcontroller further demonstrate end to end deployable realtime private and efficient HAR.

Problem

• human activity recognition
• edge computing
• spectral-temporal modeling
• resource-constrained deployment
• sensor-based recognition

Innovation

• spectral-temporal modeling
• edge-deployable architecture
• depthwise separable convolution
• channel-wise self-attention
• sensor-based human activity recognition