Privileged Foresight Distillation: Zero-Cost Future Correction for World Action Models

2026-04-28

AI Summary

This work addresses the underutilization of future information in existing world models, which typically discard their future prediction branches during inference, thereby failing to leverage training-time future observations to enhance policies that rely solely on current frames. To bridge this gap, the authors propose Privileged Foresight Distillation (PFD), a novel mechanism that formulates the action denoising correction induced by ground-truth future observations as a compressible residual signal. This signal is then distilled into a lightweight adapter within a student policy conditioned only on the current frame. PFD preserves zero inference latency and maintains compatibility with the original policy interface while effectively transferring future-conditioned knowledge. Experimental results demonstrate consistent performance gains on both LIBERO and RoboTwin benchmarks, all without requiring the generation of future video sequences.

Abstract

World action models jointly predict future video and action during training, raising an open question about what role the future-prediction branch actually plays. A recent finding shows that this branch can be removed at inference with little to no loss on common manipulation benchmarks, suggesting that future information may act merely as a regularizer on the shared visual backbone. We propose instead that joint training induces an action-conditioned correction that privileged future observations impose on action denoising, and that current-only policies capture this correction only partially. Making the account precise, we formulate privileged foresight as a residual in the action-denoising direction -- the difference between what a model predicts given the true future and what it predicts given only the current frame -- and introduce \emph{Privileged Foresight Distillation (PFD)}, which transfers this residual from a training-time teacher into a small adapter on a current-only student. The teacher and student share the same backbone and differ only in the attention mask over video tokens; future video is never generated at inference. Controlled experiments verify that this gain reflects a genuine future-conditioned correction rather than a side effect of capacity or regularization. Empirically, PFD achieves consistent improvements on LIBERO and RoboTwin manipulation benchmarks while preserving the current-only inference interface at negligible added latency. This view reframes the role of future information in world action models: not as a target to predict, nor as a regularizer to absorb, but as a compressible correction to be distilled.

Problem

• world action models
• privileged foresight
• future prediction
• action denoising
• distillation

Innovation

• Privileged Foresight Distillation
• world action models
• action-conditioned correction
• future distillation
• current-only inference