DemoBridge: A Simulation-in-the-Loop Toolkit for Single-View Human Demonstration Retargeting

2026-07-10

AI Summary

This work addresses the challenging problem of cross-embodiment retargeting from single-view human hand demonstrations to robot arm trajectories, tackling key issues such as perceptual noise, collision avoidance, and motion feasibility. The authors propose an end-to-end, collision-aware planner that jointly optimizes full joint-space trajectories, preserving the fidelity of the demonstrated path while automatically inferring grasp timing and selecting appropriate grasp poses with whole-body–object collision avoidance. The method integrates monocular RGB-based 3D perception, inverse kinematics, and closed-loop physics simulation, and supports modular configuration of perception, robot, and pipeline components. Evaluation on three real-world tasks and synthetic benchmarks demonstrates that the generated trajectories are dynamically stable, highly faithful to the demonstrations, and directly usable for downstream policy learning.

Abstract

We present DemoBridge, an toolkit that turns a single-view RGB stereo recording of a human hand demonstration into an executable, physics-validated robot-arm trajectory. Retargeting across the embodiment gap is hard. A robot arm reaches a target with a long, articulated body whose links carry far more collision volume than a hand. Solving inverse kinematics for the mapped end-effector pose often yields no collision-free solution, and a trajectory imposes this at every waypoint. A single view adds noise, leaving the demonstrated reference inaccurate. At the core of DemoBridge is a single collision-aware planner. It optimizes the whole joint trajectory at once, reasoning jointly over alternative grasp poses, whole-arm and grasped-object collision, and fidelity to the demonstrated path. A physics simulator runs in the loop. It validates each phase as it is produced and backtracks on failure, so a demonstration that cannot be reproduced as given is re-planned rather than discarded. The resulting action sequence is dynamically stable and faithful to the demonstrated manipulation. It also doubles as a ready-to-use simulation rollout for policy learning. Grasp timing is inferred automatically, and the perception backends, robot, and pipeline stages are swappable from configuration. We evaluate whole-pipeline retargeting on three real-demonstration tasks and the planner on a controlled synthetic benchmark. Our code is available at https://gitlab.kuleuven.be/u0123974/demo-bridge/ .

Problem

• retargeting
• embodiment gap
• single-view demonstration
• collision-aware planning
• robot manipulation

Innovation

• simulation-in-the-loop
• single-view retargeting
• collision-aware planning
• embodiment gap
• physics-validated trajectory