FlowR2A: Learning Reward-to-Action Distribution for Multimodal Driving Planning

2026-06-23

AI Summary

This work addresses the tension in multimodal driving planning between the fixed action vocabulary of scoring-based methods and the sparse supervision inherent in anchor-based approaches. To reconcile these limitations, the authors propose a reward-conditioned generative modeling paradigm that reformulates simulation rewards as generation conditions. A flow-matching decoder learns a reward-conditioned action distribution from dense trajectory–reward pairs, thereby unifying dense supervision with dynamic proposal generation. By incorporating fine-grained temporal reward conditioning and reward noise augmentation, the method enables controllable sampling at test time, effectively balancing hard safety constraints with soft progress objectives. Evaluated on the NAVSIM v1/v2 benchmarks, the approach significantly outperforms existing methods, achieving state-of-the-art quality in generated multimodal driving proposals.

Abstract

Multimodal driving planning faces a long-standing tension between two paradigms: scoring-based methods benefit from dense reward supervision but are confined to a fixed action vocabulary, while anchor-based methods generate proposals dynamically yet suffer from sparse supervision constrained to a single ground-truth trajectory. In this work, we propose FlowR2A, which resolves this tension by reframing simulation-based rewards from discriminative targets into generative conditions. By learning the reward-conditioned action distribution from dense trajectory-reward pairs with a flow-matching decoder, FlowR2A unifies the dense supervision of scoring-based methods with the proposal generation of anchor-based methods in a single generative model, forcing the model to internalize the correlation between an action and its outcomes in safety, progress, comfort, and rule compliance. To balance hard safety constraints against soft progress objectives, we introduce fine-grained per-timestep reward conditioning and reward noise augmentation. The generative formulation naturally supports controllable test-time sampling via reward guidance and anchored sampling, producing high-quality proposals. FlowR2A achieves state-of-the-art results on the NAVSIM v1 and v2 benchmarks, with multimodal proposals of substantially higher quality than prior methods.

Problem

• multimodal driving planning
• reward supervision
• action distribution
• trajectory generation
• autonomous driving

Innovation

• flow-matching
• reward-conditioned generation
• multimodal driving planning
• dense reward supervision
• controllable sampling