WiFi based Human Fall and Activity Recognition using Transformer based Encoder Decoder and Graph Neural Networks

2025-04-23

AI Summary

To address privacy-sensitive fall detection in home-based elderly care, this work tackles the challenge of achieving accurate, non-intrusive human pose estimation and activity recognition without compromising user privacy—bypassing the ethical and practical limitations of vision-based surveillance. Method: We propose a novel CSI-based framework comprising (i) TED Net, a hybrid CNN-Transformer architecture that jointly models spatiotemporal features from WiFi Channel State Information (CSI) for high-fidelity skeleton estimation; and (ii) a Directional Graph Neural Network (DGNN) that classifies activities—including falls—using the estimated CSI-driven skeletal sequences. Contribution/Results: Evaluated on a public multimodal dataset and a newly collected 20-subject fall dataset, our method achieves state-of-the-art skeleton estimation accuracy among CSI-based approaches. DGNN attains fall/non-fall classification accuracy comparable to RGB-based methods while demonstrating superior robustness to environmental variations. This is the first work to demonstrate CSI-derived skeletons achieving performance on par with vision systems in fall detection, establishing a new privacy-preserving paradigm for health monitoring.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Intelligent Robots: State Estimation
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Human pose estimation and action recognition have received attention due to their critical roles in healthcare monitoring, rehabilitation, and assistive technologies. In this study, we proposed a novel architecture named Transformer based Encoder Decoder Network (TED Net) designed for estimating human skeleton poses from WiFi Channel State Information (CSI). TED Net integrates convolutional encoders with transformer based attention mechanisms to capture spatiotemporal features from CSI signals. The estimated skeleton poses were used as input to a customized Directed Graph Neural Network (DGNN) for action recognition. We validated our model on two datasets: a publicly available multi modal dataset for assessing general pose estimation, and a newly collected dataset focused on fall related scenarios involving 20 participants. Experimental results demonstrated that TED Net outperformed existing approaches in pose estimation, and that the DGNN achieves reliable action classification using CSI based skeletons, with performance comparable to RGB based systems. Notably, TED Net maintains robust performance across both fall and non fall cases. These findings highlight the potential of CSI driven human skeleton estimation for effective action recognition, particularly in home environments such as elderly fall detection. In such settings, WiFi signals are often readily available, offering a privacy preserving alternative to vision based methods, which may raise concerns about continuous camera monitoring.

Problem

• Estimating human poses from WiFi signals for healthcare monitoring
• Recognizing actions using WiFi-derived skeleton data
• Providing privacy-preserving fall detection without cameras

Innovation

• Transformer based Encoder Decoder for WiFi CSI
• Directed Graph Neural Network for action recognition
• Robust performance in fall and non fall cases