Safe reinforcement learning with online filtering for fatigue-predictive human-robot task planning and allocation in production

2026-04-14

AI Summary

This study addresses the challenge of balancing dynamic task allocation with worker fatigue and safety in human-robot collaborative manufacturing. The authors propose the PF-CD3Q method, which, for the first time, treats individual fatigue parameters as uncertain variables and integrates particle filtering to estimate physiological states online. By leveraging a Constrained Dueling Double Deep Q-Network (CD3Q), the approach enables real-time, task-level fatigue prediction. Task assignment is formulated as a constrained Markov decision process, wherein action selection explicitly excludes policies that violate personalized fatigue thresholds, thereby enabling safe, adaptive, and individualized task planning. Experimental results demonstrate that the method effectively reconciles operational efficiency with ergonomic considerations in complex production environments and significantly enhances adaptability to intra-day variations in individual fatigue sensitivity.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Learning & Optimization for ROB
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Human-robot collaborative manufacturing, a core aspect of Industry 5.0, emphasizes ergonomics to enhance worker well-being. This paper addresses the dynamic human-robot task planning and allocation (HRTPA) problem, which involves determining when to perform tasks and who should execute them to maximize efficiency while ensuring workers' physical fatigue remains within safe limits. The inclusion of fatigue constraints, combined with production dynamics, significantly increases the complexity of the HRTPA problem. Traditional fatigue-recovery models in HRTPA often rely on static, predefined hyperparameters. However, in practice, human fatigue sensitivity varies daily due to factors such as changed work conditions and insufficient sleep. To better capture this uncertainty, we treat fatigue-related parameters as inaccurate and estimate them online based on observed fatigue progression during production. To address these challenges, we propose PF-CD3Q, a safe reinforcement learning (safe RL) approach that integrates the particle filter with constrained dueling double deep Q-learning for real-time fatigue-predictive HRTPA. Specifically, we first develop PF-based estimators to track human fatigue and update fatigue model parameters in real-time. These estimators are then integrated into CD3Q by making task-level fatigue predictions during decision-making and excluding tasks that exceed fatigue limits, thereby constraining the action space and formulating the problem as a constrained Markov decision process (CMDP).

Problem

• human-robot task planning
• fatigue prediction
• safe reinforcement learning
• ergonomics
• production dynamics

Innovation

• safe reinforcement learning
• particle filter
• fatigue prediction
• human-robot task allocation
• constrained Markov decision process