World Value Models for Robotic Manipulation

2026-06-23

AI Summary

Existing robotic value models struggle to effectively capture temporal dynamics and exhibit poor performance when evaluating task progress on mixed-quality data. This work proposes the World Value Model (WVM), the first approach to integrate world models into general-purpose value modeling. By fusing historical state beliefs with future outcome planning, WVM enables high-fidelity temporal value estimation. The method substantially enhances robustness in value prediction on mixed-quality datasets and supports diverse policy extraction strategies. To facilitate evaluation, we introduce Suboptimal-Value-Bench, a new benchmark comprising 800 suboptimal trajectories. Experiments demonstrate that WVM achieves state-of-the-art performance in value ordering correlation (VOC) on both established and newly introduced benchmarks, and significantly improves policy learning outcomes in both simulated and real-world robotic tasks.

Abstract

Generalist value models play a pivotal role in scaling robotic policy learning from large-scale, mixed-quality data. Mathematically, accurate value estimation demands deep temporal understanding, requiring models to both ground the current belief using historical context and plan over future outcomes. However, most existing robotic value models are built on Vision-Language Model (VLM) backbones that are pretrained primarily on static or temporally sparse visual observations, lacking the requisite temporal modeling capabilities for value estimation. Unlike VLMs, world models naturally excel at temporal modeling and future planning, making them ideal foundations for learning generalizable value functions. Driven by this insight, we marry world models with value estimation to construct a new generalist robotic value model, World Value Model (WVM), that offers accurate task progressions to assess data quality. On standard benchmarks, WVM delivers state-of-the-art (SOTA) Value-Order Correlation (VOC) results. Complementing standard evaluation suites that contains only expert data, we further introduce Suboptimal-Value-Bench, a multi-embodiment benchmark consisting of 800 suboptimal trajectories with high-fidelity, human-labeled frame annotations. Our evaluations show that WVM maintains its SOTA performance on Suboptimal-Value-Bench, establishing its robustness in handling both expert and suboptimal data. When deployed for policy learning, WVM improves manipulation performance across various policy extraction approaches in both simulated and real-world deployment, providing robust guidance for learning from mixed-quality data.

Problem

• robotic value models
• temporal modeling
• mixed-quality data
• value estimation
• suboptimal trajectories

Innovation

• World Value Model
• temporal modeling
• value estimation
• suboptimal trajectories
• robotic manipulation