Mean Flow Policy with Instantaneous Velocity Constraint for One-step Action Generation

2026-02-14

AI Summary

This work addresses the longstanding trade-off between expressiveness and computational efficiency in flow-based reinforcement learning policies, which often struggle to enable both high-fidelity and fast single-step action generation. To overcome this limitation, we propose the Mean Velocity Policy (MVP), which models a mean velocity field to facilitate rapid single-step sampling. Crucially, MVP introduces Instantaneous Velocity Constraints (IVC) as essential boundary conditions in policy learning—a novel formulation that we theoretically prove enhances both policy expressiveness and learning accuracy. By integrating deterministic sampling, MVP achieves state-of-the-art success rates across multiple robotic manipulation tasks in Robomimic and OGBench, while simultaneously offering substantial improvements in both training and inference speed.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Learning expressive and efficient policy functions is a promising direction in reinforcement learning (RL). While flow-based policies have recently proven effective in modeling complex action distributions with a fast deterministic sampling process, they still face a trade-off between expressiveness and computational burden, which is typically controlled by the number of flow steps. In this work, we propose mean velocity policy (MVP), a new generative policy function that models the mean velocity field to achieve the fastest one-step action generation. To ensure its high expressiveness, an instantaneous velocity constraint (IVC) is introduced on the mean velocity field during training. We theoretically prove that this design explicitly serves as a crucial boundary condition, thereby improving learning accuracy and enhancing policy expressiveness. Empirically, our MVP achieves state-of-the-art success rates across several challenging robotic manipulation tasks from Robomimic and OGBench. It also delivers substantial improvements in training and inference speed over existing flow-based policy baselines.

Problem

• reinforcement learning
• flow-based policy
• action generation
• expressiveness
• computational efficiency

Innovation

• Mean Velocity Policy
• Instantaneous Velocity Constraint
• One-step Action Generation
• Flow-based Policy
• Reinforcement Learning