A Human-in-the-Loop Confidence-Aware Failure Recovery Framework for Modular Robot Policies

2026-02-10

AI Summary

This work proposes a human-in-the-loop intelligent fault recovery framework for modular robots operating in unstructured human-robot environments, where module-level failures can lead to system breakdown and indiscriminate求助 imposes excessive burden on users. By quantifying uncertainties in perception, planning, and control modules and integrating a human intervention cost model, the framework decouples module selection from query decision-making to adaptively determine whether and from which module to request human assistance. Evaluated in both simulated environments and a real-world assistive feeding robot system, the approach significantly improves fault recovery success rates while reducing users’ cognitive and physical workload, demonstrating particular efficacy for individuals with mobility impairments.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Robots operating in unstructured human environments inevitably encounter failures, especially in robot caregiving scenarios. While humans can often help robots recover, excessive or poorly targeted queries impose unnecessary cognitive and physical workload on the human partner. We present a human-in-the-loop failure-recovery framework for modular robotic policies, where a policy is composed of distinct modules such as perception, planning, and control, any of which may fail and often require different forms of human feedback. Our framework integrates calibrated estimates of module-level uncertainty with models of human intervention cost to decide which module to query and when to query the human. It separates these two decisions: a module selector identifies the module most likely responsible for failure, and a querying algorithm determines whether to solicit human input or act autonomously. We evaluate several module-selection strategies and querying algorithms in controlled synthetic experiments, revealing trade-offs between recovery efficiency, robustness to system and user variables, and user workload. Finally, we deploy the framework on a robot-assisted bite acquisition system and demonstrate, in studies involving individuals with both emulated and real mobility limitations, that it improves recovery success while reducing the workload imposed on users. Our results highlight how explicitly reasoning about both robot uncertainty and human effort can enable more efficient and user-centered failure recovery in collaborative robots. Supplementary materials and videos can be found at: http://emprise.cs.cornell.edu/modularhil

Problem

• human-in-the-loop
• failure recovery
• modular robot policies
• human workload
• uncertainty

Innovation

• human-in-the-loop
• modular robot policies
• failure recovery
• uncertainty calibration
• human workload minimization