Stream-R1: Reliability-Perplexity Aware Reward Distillation for Streaming Video Generation

2026-05-05

AI Summary

Existing distillation-based approaches for streaming video generation treat all rollouts, frames, and pixels uniformly, overlooking critical variations in the reliability of supervision signals across rollouts and the spatiotemporal complexity of different regions, thereby limiting distillation efficacy. To address this, we propose a reliability–perplexity-aware reward distillation framework that jointly models rollout-level reliability and spatiotemporal optimization potential for the first time. Our method introduces a dual adaptive weighting mechanism grounded in a pretrained video reward model, complemented by a multidimensional quality balancing strategy and distribution-matching distillation. Notably, it achieves substantial improvements in visual fidelity, motion coherence, and text alignment on standard benchmarks without altering the model architecture or incurring additional inference overhead.

Abstract

Distillation-based acceleration has become foundational for making autoregressive streaming video diffusion models practical, with distribution matching distillation (DMD) as the de facto choice. Existing methods, however, train the student to match the teacher's output indiscriminately, treating every rollout, frame, and pixel as equally reliable supervision. We argue that this caps distilled quality, since it overlooks two complementary axes of variance in DMD supervision: Inter-Reliability across student rollouts whose supervision varies in reliability, and Intra-Perplexity across spatial regions and temporal frames that contribute unequally to where quality can still be improved. The objective thus conflates two questions under a uniform weight: whether to learn from each rollout, and where to concentrate optimization within it. To address this, we propose Stream-R1, a Reliability-Perplexity Aware Reward Distillation framework that adaptively reweights the distillation objective at both rollout and spatiotemporal-element levels through a single shared reward-guided mechanism. At the Inter-Reliability level, Stream-R1 rescales each rollout's loss by an exponential of a pretrained video reward score, so that rollouts with reliable supervision dominate optimization. At the Intra-Perplexity level, it back-propagates the same reward model to extract per-pixel gradient saliency, which is factored into spatial and temporal weights that concentrate optimization pressure on regions and frames where refinement yields the largest expected gain. An adaptive balancing mechanism prevents any single quality axis from dominating across visual quality, motion quality, and text alignment. Stream-R1 attains consistent improvements on all three dimensions over distillation baselines on standard streaming video generation benchmarks, without architectural modification or additional inference cost.

Problem

• reward distillation
• streaming video generation
• reliability
• perplexity
• distribution matching distillation

Innovation

• Reward Distillation
• Reliability-Aware Learning
• Perplexity-Aware Optimization
• Streaming Video Generation
• Distribution Matching Distillation