Behavior Uncloning: Distilling Mode Redirection into Policy Weights without Inference-Time Steering

2026-06-28

AI Summary

Behavioral cloning often inherits unsafe or undesirable behavioral modes from demonstration data. To address this, this work proposes MoRE, a method that incorporates a brief “anti-cloning” phase to distill redirection signals—provided by a pattern classifier—into the policy weights, thereby guiding the policy toward desired behaviors without incurring additional inference overhead. MoRE is the first approach to eliminate undesirable modes without requiring intervention during inference, while preserving task performance, offering both efficiency and broad applicability. Compatible with architectures such as Diffusion Policy and Pi0.5 VLA, MoRE leverages classifier-guided distillation and a retention loss design, achieving an average 44-percentage-point improvement in deployment success across eight simulated and real-world tasks. Its performance approaches that of retraining baselines while maintaining original inference speed and task capabilities.

Abstract

Behavior-cloned policies often learn multiple behavior modes from demonstration datasets, including modes that are unsafe or otherwise undesired at deployment. For example, a policy trained on diverse handover demonstrations may learn to pass a knife blade-first. Standard remedies such as data curation and inference-time steering either require access to the original demonstrations for full retraining or add substantial inference-time overhead. To address this gap, we propose MoRE(Mode Redirection), which redirects policy rollouts toward desired behavior modes through a short "uncloning" step. Specifically, MoRE distills the redirection signal from a temporary mode classifier into the policy weights to steer behavior. A retain loss balances this edit by preserving desired-mode competence, allowing the standalone policy to suppress unwanted modes with zero inference-time overhead. Across eight simulated and real-world tasks, MoRE improves the average deployment success rate (SR) by 44 percentage points over the original mixed-mode policy. Among all compared adaptation and steering baselines, MoRE achieves the strongest SR and approaches the filtered-data retraining reference, while preserving task competence and inference speed. MoRE also generalizes across robot policy backbones, including Diffusion Policy and the Pi0.5 VLA, diverse task categories, and real-world deployments.

Problem

• behavior cloning
• undesired behavior modes
• policy adaptation
• inference-time overhead
• safe deployment

Innovation

• Mode Redirection
• Behavior Uncloning
• Policy Distillation
• Zero Inference Overhead
• Robot Policy Adaptation