Distilling Physical Priors into Streaming World Models

2026-08-08

AI Summary

This work addresses the physical inconsistency often exhibited by streaming world models in long-horizon prediction, which frequently violates fundamental physical laws. To mitigate this issue, the authors propose PhyS, a three-stage framework: first, they construct PhyS-120K, a large-scale dataset of real-world physical interaction videos; second, they perform physics-aware fine-tuning on a bidirectional DiT teacher model and distill it into a lightweight causal DiT suitable for streaming generation; third, they incorporate online reinforcement learning with a Temporal Credit Routing (TCR) mechanism to further enhance physical plausibility. This end-to-end approach uniquely integrates real physical data, physics-aware fine-tuning, causal distillation, and TCR, achieving state-of-the-art performance—surpassing Wan2.1-14B by 18.2% on PhysicsIQ and outperforming Self-, Rolling-, and Causal Forcing baselines by 23.7%, 14.8%, and 31.4%, respectively—while also demonstrating significant gains on benchmarks such as VideoPhy.

Abstract

Streaming world models predict future visual states online while maintaining physically coherent dynamics over long horizons. However, their rollouts often violate basic physical constraints. A common approach distills pretrained bidirectional DiTs into few-step causal generators. However, this paradigm suffers from two fundamental limitations: generic bidirectional teachers acquire limited physical priors from visually oriented pretraining, and the limited priors suffer further loss during bidirectional-to-causal distillation. We present PhyS, a three-stage framework for distilling physical priors into streaming world models. To acquire physical priors from real-world interactions, we construct PhyS-120K, a dataset of 120K real-world physical-interaction videos spanning rigid-body dynamics, soft-body deformation, fluid phenomena, and phase transitions. Each video is annotated with structured descriptions of object properties and causal state transitions. Physics-aware supervised fine-tuning injects the physical priors into a bidirectional 14B DiT teacher, which we then distill into a lightweight 1.3B causal DiT for few-step autoregressive streaming generation. Finally, we use online reinforcement learning to incentivize the distilled model to generate physically plausible rollouts and further propose Temporal Credit Routing (TCR) to address temporal credit assignment. TCR evaluates physical consistency over overlapping temporal windows and routes the resulting group-relative advantages to temporally aligned denoising actions. On PhysicsIQ, PhyS improves the Wan2.1-14B teacher by 18.2\% and the Self Forcing, Rolling Forcing, and Causal Forcing by 23.7\%, 14.8\%, and 31.4\%, respectively. Results also improve the physics-aware video benchmarks VideoPhy, VideoPhy2, and PhyGenBench. The dataset, code, and more sample videos are available on our Project Page.

Problem

• streaming world models
• physical priors
• physical consistency
• causal generation
• video prediction

Innovation

• physical priors
• streaming world models
• distillation
• Temporal Credit Routing
• DiT