Learning Human Reaching Optimality Principles from Minimal Observation Inverse Reinforcement Learning

2025-09-30

AI Summary

This study investigates the dynamic, cross-subject-consistent cost structure underlying human arm reaching movements, specifically focusing on time-varying cost weights. We propose the Minimum-Observation Inverse Reinforcement Learning (MO-IRL) framework, which integrates a planar two-link biomechanical model with maximum-entropy IRL to learn piecewise-linear time-varying cost functions from minimal demonstration data. Trajectory segmentation and iterative cost-weight optimization are performed using high-fidelity motion-capture data. Results show that, after only ten training trials, joint-angle RMSEs are 6.4° and 5.6° for six- and eight-segment cost functions, respectively—substantially outperforming the static-weight baseline (10.4°). Cross-subject validation yields an average RMSE of ~8°, demonstrating unprecedented generalization in MO-IRL. Our key contribution is a novel, interpretable, low-sample-cost modeling paradigm for dynamic motor costs that exhibits strong robustness across subjects.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Learning & Optimization for ROB
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This paper investigates the application of Minimal Observation Inverse Reinforcement Learning (MO-IRL) to model and predict human arm-reaching movements with time-varying cost weights. Using a planar two-link biomechanical model and high-resolution motion-capture data from subjects performing a pointing task, we segment each trajectory into multiple phases and learn phase-specific combinations of seven candidate cost functions. MO-IRL iteratively refines cost weights by scaling observed and generated trajectories in the maximum entropy IRL formulation, greatly reducing the number of required demonstrations and convergence time compared to classical IRL approaches. Training on ten trials per posture yields average joint-angle Root Mean Squared Errors (RMSE) of 6.4 deg and 5.6 deg for six- and eight-segment weight divisions, respectively, versus 10.4 deg using a single static weight. Cross-validation on remaining trials and, for the first time, inter-subject validation on an unseen subject's 20 trials, demonstrates comparable predictive accuracy, around 8 deg RMSE, indicating robust generalization. Learned weights emphasize joint acceleration minimization during movement onset and termination, aligning with smoothness principles observed in biological motion. These results suggest that MO-IRL can efficiently uncover dynamic, subject-independent cost structures underlying human motor control, with potential applications for humanoid robots.

Problem

• Modeling human arm movements with time-varying cost functions
• Reducing demonstration data requirements in inverse reinforcement learning
• Discovering dynamic cost structures for humanoid robot applications

Innovation

• MO-IRL learns phase-specific cost functions from minimal data
• It segments trajectories and scales them in maximum entropy formulation
• The method uncovers dynamic subject-independent motor control principles