Robust Assembly State Reasoning from Action Recognition for Human-Robot Collaboration

2026-06-18

AI Summary

This study addresses the challenge of robustly inferring task states in human-robot collaborative assembly under realistic conditions such as sensor noise and repetitive actions. The authors systematically evaluate multiple state-tracking approaches—namely logic-based rules, Hidden Markov Models (HMMs), and neural networks (NNs)—using action recognition outputs across multiple datasets and varying noise levels. Their analysis delineates the operational boundaries of each method, revealing that explicitly modeling action duration is critical for handling repetitive actions. Based on these findings, they propose a task-dependent strategy for selecting the most suitable tracking approach. Experimental results demonstrate that HMMs and NNs excel in low-variability tasks, whereas logic-based methods exhibit superior robustness in high-noise or complex scenarios.

Abstract

Human Action Recognition (HAR) is frequently investigated in Human-Robot Collaboration (HRC) research to understand what actions have been performed and hence the state of a collaborative task. Accurately tracking an assembly state from HAR is however not fully investigated, and in realistic scenarios is not a trivial task. This research systematically investigates and compares methods for tracking assembly state using action recognition inputs. Investigations using two diverse datasets and five state tracking approaches, including logic-based, Hidden Markov Model (HMM), and neural network (NN) methods, show that optimal approaches are not uniform across different tasks and that different methods fail under different circumstances. Testing is performed using both simulated inputs with varying noise levels and realistic inputs from a HAR model. Results show NN and HMM methods can perform well in tasks with limited variability, but for other scenarios logic-based approaches can be more robust. Methods which model expected action duration are also important for tasks with repeated actions where no additional sensing is provided.

Problem

• Human-Robot Collaboration
• Human Action Recognition
• Assembly State Tracking
• Robust Reasoning
• State Estimation

Innovation

• Human Action Recognition
• Assembly State Tracking
• Human-Robot Collaboration
• Hidden Markov Model
• Logic-based Reasoning