Temporal Difference Calibration in Sequential Tasks: Application to Vision-Language-Action Models

2026-04-22

AI Summary

This work addresses the insufficient uncertainty calibration of vision-language-action (VLA) models in partially observable sequential tasks by proposing a novel sequential calibration framework that establishes, for the first time, a theoretical connection between sequential calibration and reinforcement learning value functions. The approach integrates temporal difference (TD) value estimation with a sequential extension of the Brier score to enable dynamic, time-evolving calibration, and demonstrates that single-step action probabilities can effectively reflect task-level uncertainty. Experimental results show that the proposed framework significantly outperforms existing calibration methods on both simulated and real-world robotic datasets, simultaneously improving both the calibration quality and task success rates of VLA models.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Sequential Decision Making
• Computer Vision: Language and Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recent advances in vision-language-action (VLA) models for robotics have highlighted the importance of reliable uncertainty quantification in sequential tasks. However, assessing and improving calibration in such settings remains mostly unexplored, especially when only partial trajectories are observed. In this work, we formulate sequential calibration for episodic tasks, where task-success confidence is produced along an episode, while success is determined at the end of it. We introduce a sequential extension of the Brier score and show that, for binary outcomes, its risk minimizer coincides with the VLA policy's value function. This connection bridges uncertainty calibration and reinforcement learning, enabling the use of temporal-difference (TD) value estimation as a principled calibration mechanism over time. We empirically show that TD calibration improves performance relative to the state-of-the-art on simulated and real-robot data. Interestingly, we show that when calibrated using TD, the VLA's single-step action probabilities can yield competitive uncertainty estimates, in contrast to recent findings that employed different calibration techniques.

Problem

• Temporal Difference Calibration
• Sequential Tasks
• Vision-Language-Action Models
• Uncertainty Quantification
• Task-Success Confidence

Innovation

• Temporal Difference Calibration
• Sequential Calibration
• Vision-Language-Action Models
• Brier Score
• Uncertainty Quantification