Decision-Level Fusion for Robust Wearable Affect Recognition

2026-05-14

AI Summary

This study addresses the challenge of insufficient robustness in affect recognition on wearable devices under real-world conditions, where non-stationary physiological signals, artifacts, and missing sensor data degrade performance. To tackle this, the authors propose a method that extracts transient affective patterns through non-stationary feature representation and reliability-weighted fusion. Specifically, Fourier–Bessel series expansion (FBSE) combined with empirical wavelet transform (EWT) is employed to capture transient spectral features from multimodal physiological signals—including ECG, EDA, BVP, EMG, and ACC—and a decision-level fusion mechanism is designed based on prediction uncertainty and modality reliability. Experiments on the WESAD dataset demonstrate that the proposed approach significantly outperforms feature-level fusion in approximately 48% of cases and matches or exceeds its performance in 84% of cases, thereby enhancing robustness under partial sensor failure or strong noise interference.

Abstract

Automatic recognition of affective state from wearable physiology has clear societal impact for public health, preventive care, and stress-aware interventions, but real deployments require robustness to non-stationary dynamics, artefacts, and missing sensors. We study this problem on WESAD, using baseline, stress, and amusement conditions, where common fixed-basis spectral features such as FFT bandpower and Welch PSD can oversmooth short-lived discriminative patterns. We propose a non-stationary pipeline that combines Fourier-Bessel Series Expansion (FBSE) with EWT data-driven spectral segmentation to extract mode-wise transient descriptors. For multimodal integration, we adopt decision-level aggregation over per-modality predictors and weight each modality by predictive uncertainty and modality reliability. Results on WESAD, using 15 subjects and ECG, EDA, BVP, EMG, and ACC signals across three classes, indicate that decision-level aggregation is approximately 84 percent of the time at least as good as feature-level aggregation, and approximately 48 percent of the time strictly better, suggesting improved robustness under heterogeneous and partially reliable sensing.

Problem

• affect recognition
• wearable physiology
• non-stationary dynamics
• sensor artefacts
• missing sensors

Innovation

• Fourier-Bessel Series Expansion
• Empirical Wavelet Transform
• decision-level fusion
• non-stationary signal processing
• affect recognition