Perfect Demo Makes Poor Teacher: Learning Robust Alignment from Critical Motion Segments

2026-06-14

AI Summary

This work addresses the limited effectiveness of smooth expert demonstrations in fine-grained alignment tasks, where critical correction phases often lack sufficient supervisory signal, thereby constraining imitation learning performance. To overcome this, the authors propose STAIR, a method that identifies and resamples pivotal motion segments that determine task success or failure, and integrates them with spatiotemporal representation learning to distill short-horizon motion cues into dense, motion-aware supervision signals. Relying solely on fluent demonstration data, STAIR effectively bridges vision-language models with action expert modules while expanding action-state coverage. Experimental results demonstrate that the approach improves task success rates from 50.0% to 62.2%, approaching the 64.4% achieved with deliberately slow-motion demonstrations, and substantially enhances policy robustness in alignment-intensive tasks.

Abstract

Expert demonstrations are widely assumed to be the gold standard for robot imitation learning. Yet for fine-grained manipulation such as insertion, stacking, and alignment, we uncover a counterintuitive failure mode: fluent demonstrations can be poor teachers. A skilled teleoperator compresses the decisive moments of alignment and recovery into a brief temporal window, leaving the policy flooded with redundant free-space motion and starved of supervision exactly where precision determines success. We address this bottleneck at two levels. At the data level, slowing down near alignment and resampling critical segments both help, yet the gain comes mainly from broadening the coverage of recovery states the policy must learn, not from reweighting frames it already has. Such data-side fixes, however, leave the policy's per-frame view untouched: a single image still maps directly to an action, and the local motion that governs correction stays implicit. We therefore turn to the representation level and introduce STAIR (\textbf{S}patio-\textbf{T}emporal feature \textbf{A}s an \textbf{I}nterface for \textbf{R}obot learning), a compact dynamic feature that bridges the vision-language model and the action expert, distilling the short-horizon motion already recorded in each trajectory into dense, motion-aware supervision. Trained on fluent data alone, STAIR recovers most of the deliberate-demonstration gain ($50.0$ to $62.2\%$ overall, approaching the $64.4\%$ of deliberate demonstrations). These results call for a more pedagogical view of robot data, optimized for machine learnability rather than human efficiency alone.

Problem

• imitation learning
• expert demonstrations
• fine-grained manipulation
• critical motion segments
• robot learning

Innovation

• imitation learning
• critical motion segments
• spatio-temporal representation
• STAIR
• robotic manipulation