Feasibility-aware Imitation Learning from Observations through a Hand-mounted Demonstration Interface

2025-03-12

AI Summary

This work addresses the challenge that human demonstrations often contain kinematically or dynamically infeasible actions for robots due to human–robot morphological and dynamic disparities. To tackle this, we propose Feasibility-Aware Behavioral Cloning (FABCO), a framework that embeds a robot dynamics model into the teaching loop to enable real-time feasibility assessment and visual feedback, coupled with a feasibility-weighted behavioral cloning learning mechanism. Technically, FABCO integrates pre-trained forward and inverse dynamics models, a wearable hand motion capture interface, and NASA-TLX-based subjective workload evaluation. Experimental validation on a pipette-insertion task demonstrates that FABCO significantly improves policy robustness and data efficiency. Four human participants confirmed that its feedback mechanism substantially reduces cognitive load (p < 0.01) and enhances demonstration quality. The core contribution lies in the first integration of a feasibility-aware closed-loop into the imitation learning teaching pipeline, enabling online compensation for human–robot motion characteristic mismatches.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Imitation learning through a demonstration interface is expected to learn policies for robot automation from intuitive human demonstrations. However, due to the differences in human and robot movement characteristics, a human expert might unintentionally demonstrate an action that the robot cannot execute. We propose feasibility-aware behavior cloning from observation (FABCO). In the FABCO framework, the feasibility of each demonstration is assessed using the robot's pre-trained forward and inverse dynamics models. This feasibility information is provided as visual feedback to the demonstrators, encouraging them to refine their demonstrations. During policy learning, estimated feasibility serves as a weight for the demonstration data, improving both the data efficiency and the robustness of the learned policy. We experimentally validated FABCO's effectiveness by applying it to a pipette insertion task involving a pipette and a vial. Four participants assessed the impact of the feasibility feedback and the weighted policy learning in FABCO. Additionally, we used the NASA Task Load Index (NASA-TLX) to evaluate the workload induced by demonstrations with visual feedback.

Problem

• Addresses robot policy learning from human demonstrations
• Ensures robot-executable actions via feasibility assessment
• Improves data efficiency and policy robustness

Innovation

• Feasibility-aware behavior cloning from observation
• Visual feedback refines human demonstrations
• Weighted policy learning improves data efficiency