Temporal Posed and Spontaneous Gesture Recognition from Electromyography in the Rock-Paper-Scissors Game

2026-06-28

AI Summary

This study addresses the challenge of distinguishing between premeditated and spontaneous hand gestures in real-time multi-user interaction scenarios using electromyographic (EMG) signals. Employing dual-channel surface EMG from the forearm during a “rock–paper–scissors” task, the work analyzes temporal characteristics of gesture execution and demonstrates for the first time that user intent can be detected from EMG signals up to 800 milliseconds before visible movement onset. Furthermore, it reveals that reactive EMG responses in an opponent’s muscles contain discernible information about the observed gesture. Through temporal analysis of EMG onset and peak features combined with machine learning models, the system achieves 63.4% accuracy in identifying premeditated gestures and 53.6% when generalized to spontaneous gestures. Notably, gesture recognition from the opponent’s EMG peaks reaches 65% accuracy, albeit lagging visual action by 2082 milliseconds, thereby opening a novel pathway for interactive intention recognition.

Abstract

The importance of gesture recognition has been acknowledged in many domains requiring real-time recognition systems. Two requirements for these are fast recognition in multiuser contexts. Therefore, we explored the temporal characteristics of electromyography (EMG) and its accuracy in recognizing gestures in a Rock-Paper-Scissors (RPS) game. Twenty-four participants played RPS in dyads, while a two-channel EMG was recorded from the forearm. We found out that EMG onsets could be detected at least 800 ms before the gesture's visible onset, and that the EMG peaks around 342 ms before the visible onset of the gesture. Furthermore, we evaluated self-gesture recognition in both posed and spontaneous gesture conditions. The mean accuracy for posed gestures reached 63.4%. The model trained on posed gestures achieved 53.6% for spontaneous gestures, with considerable variation across individuals. We also checked whether detecting a player's gesture from the opponent's EMG was possible. The peak mean accuracy was 65%, peaking at 2082 ms after the visual onset of the gesture. This suggests that the opponent's reaction to an observed gesture contains information about the observed gesture due to the dynamics of the interactions while playing. The temporal predictive advantage of EMG signals, where muscle activation precedes observable movement, offers potential benefits for applications requiring rapid intent recognition, such as human-computer interaction and assistive technologies. Future work should focus on refining onset detection and reducing the impact of spontaneous movement variability across conditions to improve recognition performance in dynamic and real-world environments.

Problem

• gesture recognition
• electromyography
• spontaneous gesture
• posed gesture
• temporal prediction

Innovation

• electromyography
• gesture recognition
• temporal prediction
• spontaneous gesture
• human-computer interaction