Robot-Gated Interactive Imitation Learning with Adaptive Intervention Mechanism

2025-06-10

AI Summary

Interactive Imitation Learning (IIL) suffers from high human supervision overhead and substantial expert demonstration requirements. Method: This paper proposes Adaptive Intervention Mechanism (AIM), a robot-gated framework that integrates human-in-the-loop online learning with autonomous intervention gating. AIM models human intervention decisions via a novel agent Q-function, enabling dynamic intervention based on real-time alignment between human and robot actions; it further explicitly identifies safety-critical states—an innovation in IIL—to enhance demonstration quality and safety. Contribution/Results: Experiments show that AIM reduces human takeover frequency and monitoring duration by 40% compared to Thrifty-DAgger, while significantly decreasing environment interactions and required expert demonstrations. It maintains competitive learning performance and substantially improves human-robot collaboration efficiency.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Interactive Imitation Learning (IIL) allows agents to acquire desired behaviors through human interventions, but current methods impose high cognitive demands on human supervisors. We propose the Adaptive Intervention Mechanism (AIM), a novel robot-gated IIL algorithm that learns an adaptive criterion for requesting human demonstrations. AIM utilizes a proxy Q-function to mimic the human intervention rule and adjusts intervention requests based on the alignment between agent and human actions. By assigning high Q-values when the agent deviates from the expert and decreasing these values as the agent becomes proficient, the proxy Q-function enables the agent to assess the real-time alignment with the expert and request assistance when needed. Our expert-in-the-loop experiments reveal that AIM significantly reduces expert monitoring efforts in both continuous and discrete control tasks. Compared to the uncertainty-based baseline Thrifty-DAgger, our method achieves a 40% improvement in terms of human take-over cost and learning efficiency. Furthermore, AIM effectively identifies safety-critical states for expert assistance, thereby collecting higher-quality expert demonstrations and reducing overall expert data and environment interactions needed. Code and demo video are available at https://github.com/metadriverse/AIM.

Problem

• Reducing human cognitive load in Interactive Imitation Learning
• Learning adaptive criteria for requesting human demonstrations
• Improving safety and efficiency in robot control tasks

Innovation

• Adaptive robot-gated intervention mechanism
• Proxy Q-function for alignment assessment
• Reduces expert monitoring by 40%