Stream Forcing: Constructing Unified Training Trajectory for Robust Streaming Video Generation

2026-08-10

AI Summary

This work addresses the inconsistency between training and inference noise scheduling in streaming video generation by proposing a unified training framework termed Stream Forcing. The method models diffusion sampling as a frame-indexed stochastic process parameterized by noise levels, establishing a continuous training trajectory that bridges independent sampling during training and inference-consistent sampling at test time. Through a joint calibration strategy coupled with a temporally coherent sampling algorithm, the framework ensures smoothness along the training trajectory and strong inter-frame temporal consistency, effectively balancing comprehensive training coverage with efficient inference within a single architecture. Empirically, the approach reduces FVD by 36.6% on UCF-101 and achieves a 27.9% FVD improvement in zero-shot long-horizon video extrapolation tasks.

Abstract

Streaming video generation holds strong potential for world modeling, where future frames must be inferred online sequentially to form a continuous video stream. However, streaming video diffusion models introduce a fundamental train-inference mismatch: inference follows a specialized denoising order, whereas advanced training strategies typically require diverse noise-level configurations. To address this trade-off between train-inference consistency and training coverage, we reformulate the video diffusion sampling as a frame-indexed stochastic process over noise levels. Within this stochastic process space, we construct a continuous training trajectory along which the sampling schedule progressively evolves from independent sampling to inference-consistent sampling. We further introduce a joint calibration algorithm and a temporal correlative sampling algorithm to ensure trajectory smoothness and cross-frame correlation. Building on these designs, we propose Stream Forcing, a unified training framework for streaming video generation that balances training sufficiency and inference efficiency. Extensive experiments demonstrate that Stream Forcing significantly improves generation quality with a 36.6% FVD improvement on the UCF-101 benchmark. Furthermore, our method facilitates robust zero-shot extrapolation to long-horizon video generation with a 27.9% FVD improvement on the UCF-101 benchmark.

Problem

• streaming video generation
• train-inference mismatch
• diffusion models
• noise-level configuration
• video diffusion sampling

Innovation

• Stream Forcing
• Streaming Video Generation
• Diffusion Models
• Train-Inference Consistency
• Temporal Correlation