Improving mmWave based Hand Hygiene Monitoring through Beam Steering and Combining Techniques

2025-03-21

AI Summary

To address the low accuracy and severe angular sensitivity of millimeter-wave-based hand-hygiene monitoring at long standoff distances (1.5 m), this paper proposes BeaMsteerX (BMX), a novel approach integrating intelligent beam steering with deep multi-view signal feature fusion. BMX introduces a beam-scanning-driven acquisition mechanism for multi-angle radar signals and designs a lightweight time-frequency–spatial joint feature fusion network, significantly enhancing robustness in low-SNR and large-angle (≤30°) conditions. Experimental results demonstrate that BMX achieves 91% hand-hygiene action recognition accuracy at 1.5 m—outperforming state-of-the-art methods by 31–43%. Moreover, its performance degrades by only 5% under 30° angular deviation, enabling, for the first time, contactless, end-to-end monitoring of the WHO’s six-step hand-rubbing procedure.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Multi-modal Vision

Application Category

• Security and Privacy: Large-scale security measurements
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

We introduce BeaMsteerX (BMX), a novel mmWave hand hygiene gesture recognition technique that improves accuracy in longer ranges (1.5m). BMX steers a mmWave beam towards multiple directions around the subject, generating multiple views of the gesture that are then intelligently combined using deep learning to enhance gesture classification. We evaluated BMX using off-the-shelf mmWave radars and collected a total of 7,200 hand hygiene gesture data from 10 subjects performing a six-step hand-rubbing procedure, as recommended by the World Health Organization, using sanitizer, at 1.5m -- over five times longer than in prior works. BMX outperforms state-of-the-art approaches by 31--43% and achieves 91% accuracy at boresight by combining only two beams, demonstrating superior gesture classification in low SNR scenarios. BMX maintained its effectiveness even when the subject was positioned 30 degrees away from the boresight, exhibiting a modest 5% drop in accuracy.

Problem

• Enhancing hand hygiene gesture recognition accuracy at longer ranges
• Combining multiple mmWave beam views for improved classification
• Maintaining effectiveness in low SNR and off-boresight scenarios

Innovation

• Beam steering enhances gesture recognition accuracy
• Deep learning combines multiple gesture views
• Works effectively at longer ranges and angles