Probabilistic Human Intent Prediction for Mobile Manipulation: An Evaluation with Human-Inspired Constraints

2025-07-14

AI Summary

To address inaccurate human intent inference and consequent control conflicts in human-robot collaboration, this paper proposes GUIDER, a novel two-stage probabilistic framework that enables the first joint modeling of navigation and manipulation intents. Methodologically, GUIDER employs a dual-coupled belief layer architecture integrating multi-view scanning, geometry-aware grasp feasibility testing, and end-effector kinematic constraints. A Synergy Map unifies velocity fields and occupancy grids, while dynamic intent updating leverages U2Net/FastSAM-based saliency detection and 3D grasp evaluation. Evaluated over 25 trials in Isaac Sim, GUIDER achieves navigation and manipulation intent prediction stability of 93–100% and 94–100%, respectively—improving upon baselines by 39.5%. Its prediction speed is three times faster than Trajectron. Crucially, GUIDER supports real-time, constraint-free, target-agnostic human-robot collaboration without predefined goals.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Accurate inference of human intent enables human-robot collaboration without constraining human control or causing conflicts between humans and robots. We present GUIDER (Global User Intent Dual-phase Estimation for Robots), a probabilistic framework that enables a robot to estimate the intent of human operators. GUIDER maintains two coupled belief layers, one tracking navigation goals and the other manipulation goals. In the Navigation phase, a Synergy Map blends controller velocity with an occupancy grid to rank interaction areas. Upon arrival at a goal, an autonomous multi-view scan builds a local 3D cloud. The Manipulation phase combines U2Net saliency, FastSAM instance saliency, and three geometric grasp-feasibility tests, with an end-effector kinematics-aware update rule that evolves object probabilities in real-time. GUIDER can recognize areas and objects of intent without predefined goals. We evaluated GUIDER on 25 trials (five participants x five task variants) in Isaac Sim, and compared it with two baselines, one for navigation and one for manipulation. Across the 25 trials, GUIDER achieved a median stability of 93-100% during navigation, compared with 60-100% for the BOIR baseline, with an improvement of 39.5% in a redirection scenario (T5). During manipulation, stability reached 94-100% (versus 69-100% for Trajectron), with a 31.4% difference in a redirection task (T3). In geometry-constrained trials (manipulation), GUIDER recognized the object intent three times earlier than Trajectron (median remaining time to confident prediction 23.6 s vs 7.8 s). These results validate our dual-phase framework and show improvements in intent inference in both phases of mobile manipulation tasks.

Problem

• Predict human intent for robot collaboration without constraints
• Estimate navigation and manipulation goals probabilistically in real-time
• Improve intent inference stability in mobile manipulation tasks

Innovation

• Dual-phase probabilistic framework for intent estimation
• Synergy Map blends velocity with occupancy grid
• Combines saliency detection with geometric grasp-feasibility