Visual Verification Enables Inference-time Steering and Autonomous Policy Improvement

2026-06-16

AI Summary

Current robotic systems lack the ability to autonomously refine their policies during inference. This work proposes VERITAS, a framework that, for the first time, integrates a gradient-free visual verifier with a pretrained general-purpose policy to enable real-time action evaluation and correction during deployment—without requiring additional training or human intervention. By leveraging verification trajectories for offline self-improvement, the method substantially outperforms the original general policy. Moreover, through iterative fine-tuning on self-generated data, the system demonstrates continuous performance gains, ultimately achieving efficiency approaching that of expert demonstrations.

Abstract

Robots deployed in the real world should learn from their experience and improve over time. This requires a mechanism of practicing and learning from feedback. In this paper, we propose VERITAS, a generator-verifier framework for generalist robot policies for inference-time policy steering and self-improvement. We use a pre-trained generalist robot policy as a ``generator'' and pair it with a gradient-free ``visual verifier'' that evaluates actions at inference time. This framework enables inference-time steering that improves policy performance without additional training. We demonstrate that inference-time verification consistently outperforms vanilla generalists without training on additional demonstration data. Additionally, we demonstrate that the verified rollouts provide effective supervision for offline policy improvement: policies fine-tuned on verified self-generated trajectories achieve consistent performance gains. Notably, we find that post-training with verified rollouts achieves comparable efficiency to expert demonstrations, while requiring no human interventions. Our results highlight inference-time verification as a practical and scalable mechanism for improving robotic policies during deployment.

Problem

• robot policy improvement
• inference-time steering
• autonomous learning
• visual verification
• self-improvement

Innovation

• inference-time steering
• visual verification
• self-improvement
• generalist robot policy
• offline policy learning