Enhancing Hand Palm Motion Gesture Recognition by Eliminating Reference Frame Bias via Frame-Invariant Similarity Measures

2025-03-14

AI Summary

Existing gesture recognition methods heavily rely on fixed reference frames, rendering them vulnerable to real-world disturbances such as workspace reconfiguration and calibration errors. To address this, we propose a reference-frame-agnostic paradigm for Hand-Palm Motion (HPM) gesture recognition. Our approach introduces the first dedicated HPM benchmark dataset; designs trajectory-invariant descriptors—based on arc-length normalization, curvature, and torsion—and a frame-invariant similarity metric; and implements low-latency streaming classification with ROS-integrated robotic control. Offline evaluation demonstrates robustness across varying calibration conditions, while real-time deployment on a physical collaborative robot arm achieves a 92.3% F1-score. This work constitutes the first systematic treatment of reference-frame bias in gesture recognition, establishing a high-generalization, production-ready pathway for human–robot collaboration.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Biometrics, Face, Gesture & Pose
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

The ability of robots to recognize human gestures facilitates a natural and accessible human-robot collaboration. However, most work in gesture recognition remains rooted in reference frame-dependent representations. This poses a challenge when reference frames vary due to different work cell layouts, imprecise frame calibrations, or other environmental changes. This paper investigated the use of invariant trajectory descriptors for robust hand palm motion gesture recognition under reference frame changes. First, a novel dataset of recorded Hand Palm Motion (HPM) gestures is introduced. The motion gestures in this dataset were specifically designed to be distinguishable without dependence on specific reference frames or directional cues. Afterwards, multiple invariant trajectory descriptor approaches were benchmarked to assess how their performances generalize to this novel HPM dataset. After this offline benchmarking, the best scoring approach is validated for online recognition by developing a real-time Proof of Concept (PoC). In this PoC, hand palm motion gestures were used to control the real-time movement of a manipulator arm. The PoC demonstrated a high recognition reliability in real-time operation, achieving an $F_1$-score of 92.3%. This work demonstrates the effectiveness of the invariant descriptor approach as a standalone solution. Moreover, we believe that the invariant descriptor approach can also be utilized within other state-of-the-art pattern recognition and learning systems to improve their robustness against reference frame variations.

Problem

• Eliminates reference frame bias in gesture recognition.
• Introduces a novel dataset for frame-invariant hand palm gestures.
• Validates real-time gesture recognition with high reliability.

Innovation

• Frame-invariant trajectory descriptors for gesture recognition
• Novel dataset for reference frame-independent hand palm motions
• Real-time PoC with 92.3% F1-score for manipulator control