FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space

2026-07-09

AI Summary

This work addresses the challenge that pretrained generative robotic policies often fail under distribution shifts in real-world deployment, while existing adaptation methods either require large amounts of data or rely on online reinforcement learning, compromising efficiency or safety. To overcome this, the authors propose FlowDAgger, a method that efficiently fine-tunes a frozen generative policy in latent space through human intervention. Its key innovation is an action inversion mechanism that combines inverse time integration with local noise optimization to map human-provided actions back to corresponding latent variables. These latent targets are then used to train a lightweight latent policy that guides the base generative model. Experiments demonstrate that FlowDAgger achieves superior performance over supervised fine-tuning and latent-space reinforcement learning baselines in both simulated and real-world single- and dual-arm manipulation tasks, requiring only minimal human intervention while preserving the original policy’s capabilities.

Abstract

Pretrained generative robot policies based on flow matching and diffusion have achieved impressive results across a wide range of manipulation tasks. Yet real-world deployments routinely expose failure modes outside the pretraining distribution. Closing these gaps typically requires large-scale data collection or online reinforcement learning on physical hardware, which is impractical for rapid and safe adaptation. We present FlowDAgger, a sample- and compute-efficient method for adapting frozen generative robot policies from human interventions in latent space. Our key idea is action inversion: each human expert action is mapped to the noise that would have produced it under the frozen base policy, using reverse-time integration followed by local refinement. The resulting inverted noise provides supervision for a lightweight latent policy that steers the base model at deployment time, enabling rapid skill acquisition while preserving its behavioral priors. We evaluate FlowDAgger in simulation and on real-world bimanual and single-arm manipulation, adapting both action-head VLAs and world-action models from a handful of interventions. FlowDAgger outperforms supervised fine-tuning and latent-space RL baselines and preserves pretrained skills on held-out tasks, offering a practical path for adapting robot foundation models in the real world. Website: https://microsoft.github.io/FlowDAgger

Problem

• robot policy adaptation
• out-of-distribution failures
• human-in-the-loop
• real-world deployment
• foundation models

Innovation

• FlowDAgger
• action inversion
• latent space adaptation
• human-in-the-loop
• generative robot policies