Learning Direct Control Policies with Flow Matching for Autonomous Driving

2026-05-14

AI Summary

This work addresses the challenge of generating low-latency, stable, and generalizable direct control policies for autonomous driving in out-of-distribution scenarios. The authors propose a bird’s-eye-view (BEV)-based flow matching approach, introducing flow matching for the first time into end-to-end control policy learning. By modeling a smooth vector field over geometrically centered BEV grids, the method efficiently produces executable trajectories—parameterized by acceleration and curvature—through a small number of ordinary differential equation (ODE) integration steps, enabling real-time closed-loop replanning. Experiments demonstrate that the model significantly enhances generalization and robustness in out-of-distribution environments, including multi-lane highways and unseen urban settings, successfully completing numerous driving tasks that substantially differ from the training distribution.

Abstract

We present a flow-matching planner for autonomous driving that directly outputs actionable control trajectories defined by acceleration and curvature profiles. The model is conditioned on a bird's-eye-view (BEV) raster of the surrounding scene and generates control sequences in a small number of Ordinary Differential Equations (ODE) integration steps, enabling low-latency inference suitable for real-time closed-loop re-planning. We train exclusively on urban scenarios (real urban city streets, intersections and roundabouts of the city of Parma, Italy) collected from a 2D traffic simulator with reactive agents, and evaluate in closed-loop on both in-distribution and markedly out-of-distribution environments, including multi-lane highways and unseen urban scenarios. Our results show that the model generalizes reliably to these unseen conditions, maintaining stable closed-loop control and successfully completing scenarios that differ substantially from the training distribution. We attribute this to the BEV representation, which provides a geometry-centric view of the scene that is inherently less sensitive to distributional shifts, and to the flow-matching formulation, which learns a smooth vector field that degrades gracefully under distribution shift. We provide video demonstrations of closed-loop behavior at https://marcelloceresini.github.io/DirectControlFlowMatching.

Problem

• autonomous driving
• control policies
• distribution shift
• generalization
• closed-loop control

Innovation

• flow matching
• direct control policy
• bird's-eye-view (BEV)
• ODE-based planning
• distributional robustness