MeanFlowNFT: Bringing Forward-Process RL to Average-Velocity Generators

2026-07-16

AI Summary

This work presents the first successful integration of forward-process reinforcement learning (RL) into MeanFlow, a fast generative model based on average velocity, to address the challenge of optimizing human preference rewards under few-step sampling. By constructing an instantaneous velocity predictor induced by the average velocity, we establish a mapping mechanism that adapts the RL objective from DiffusionNFT to the MeanFlow framework, enabling policy improvement guarantees without requiring backward trajectories or likelihood estimation. Experimental results demonstrate that the proposed method significantly outperforms existing few-step RL-based generators in both image and video generation, achieving state-of-the-art performance on six out of eight evaluation metrics. Notably, it attains a VBench score of 84.33 with only four sampling steps, surpassing multi-step RL approaches that use 50 steps.

Abstract

MeanFlow generators achieve fast few-step sampling by predicting average velocities over time intervals, making them attractive for efficient generation. Reinforcement learning (RL) has become a powerful way to align diffusion and flow models with human preferences and task-specific objectives. In particular, DiffusionNFT offers an efficient forward-process RL framework that does not require reverse-process trajectories or likelihood estimation. However, applying such RL methods to MeanFlow remains underexplored. DiffusionNFT optimizes instantaneous velocities, whereas MeanFlow samples with average velocities. To bridge this gap, we introduce MeanFlowNFT. Inspired by the MeanFlow identity, which bridges average and instantaneous velocities, we construct an induced instantaneous-velocity predictor. We apply the DiffusionNFT objective to this predictor, making reward optimization well-defined for MeanFlow. Sampling remains based on the average velocity, preserving MeanFlow's fast few-step generation. We further prove that MeanFlowNFT inherits DiffusionNFT's strict policy-improvement guarantee. Experiments on image and video generation show that MeanFlowNFT consistently improves baselines. Moreover, it outperforms prior state-of-the-art RL-tuned few-step generators on most metrics ($6$ of $8$ on SD3.5-M), and can even surpass multi-step RL-tuned diffusion while using only a few sampling steps. For instance, on Wan 2.1, $4$-step MeanFlowNFT reaches a VBench score of $84.33$, surpassing $50$-step LongCat-Video RL ($82.57$).

Problem

• MeanFlow
• reinforcement learning
• average velocity
• forward-process RL
• generative modeling

Innovation

• MeanFlow
• forward-process RL
• average velocity
• DiffusionNFT
• few-step generation