Personalized Emotion Detection from Floor Vibrations Induced by Footsteps

2025-03-06

AI Summary

This study addresses key challenges in emotion recognition: high privacy intrusion and limited accessibility of physiological signals, as well as the indirect, highly individualized relationship between emotions and floor vibrations. To this end, we propose EmotionVibe—a non-intrusive, privacy-preserving, personalized emotion recognition system leveraging footstep-induced floor vibrations. Our approach innovatively establishes an “emotion–gait–vibration” mapping model and constructs a set of affect-sensitive time-frequency vibration features. We further design a gait-similarity-based personalized weighted learning mechanism to mitigate inter-subject variability. Evaluated in a real-world study with 20 participants, EmotionVibe achieves mean absolute errors (MAE) of 1.11 and 1.07 for valence and arousal regression, respectively—reducing baseline MAE by 19.0% and 25.7%. This work presents the first empirical validation of floor vibration as a viable, effective modality for fine-grained emotion sensing.

Technology Category

• Cognitive Modeling & Cognitive Systems: Affective Computing
• Computer Vision: Biometrics, Face, Gesture & Pose
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Security and Privacy: Large-scale security measurements
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Emotion recognition is critical for various applications such as early detection of mental health disorders and emotion based smart home systems. Previous studies used various sensing methods for emotion recognition, such as wearable sensors, cameras, and microphones. However, these methods have limitations in long term domestic, including intrusiveness and privacy concerns. To overcome these limitations, this paper introduces a nonintrusive and privacy friendly personalized emotion recognition system, EmotionVibe, which leverages footstep induced floor vibrations for emotion recognition. The main idea of EmotionVibe is that individuals' emotional states influence their gait patterns, subsequently affecting the floor vibrations induced by their footsteps. However, there are two main research challenges: 1) the complex and indirect relationship between human emotions and footstep induced floor vibrations and 2) the large between person variations within the relationship between emotions and gait patterns. To address these challenges, we first empirically characterize this complex relationship and develop an emotion sensitive feature set including gait related and vibration related features from footstep induced floor vibrations. Furthermore, we personalize the emotion recognition system for each user by calculating gait similarities between the target person (i.e., the person whose emotions we aim to recognize) and those in the training dataset and assigning greater weights to training people with similar gait patterns in the loss function. We evaluated our system in a real-world walking experiment with 20 participants, summing up to 37,001 footstep samples. EmotionVibe achieved the mean absolute error (MAE) of 1.11 and 1.07 for valence and arousal score estimations, respectively, reflecting 19.0% and 25.7% error reduction compared to the baseline method.

Problem

• Nonintrusive emotion detection using floor vibrations
• Personalized emotion recognition via gait pattern analysis
• Overcoming privacy concerns in domestic emotion sensing

Innovation

• Nonintrusive emotion detection via floor vibrations
• Personalized system using gait pattern similarities
• EmotionVibe reduces error rates significantly