Robot Critics that Sweat the Small Stuff

2026-06-19

AI Summary

Current vision-language models struggle to discern the subtle visual differences between successful and failed executions in robotic manipulation, limiting the reliability of closed-loop control. This work proposes a fine-grained progress supervision method that leverages policy-generated pairs of success-failure trajectories to fine-tune a vision-language critic model, enabling precise progress reasoning and detection of minor failures. The approach further integrates an action-conditioned video prediction model to evaluate the visual outcomes of candidate actions, thereby selecting the optimal one. To our knowledge, this is the first effective integration of such a critic mechanism into a real-world robotic decision-making loop. Experiments demonstrate significant improvements over existing progress-reasoning baselines, with an average policy success rate increase of 11% in real-world tasks and 5.9% in simulation.

Abstract

Large vision-language models contain several priors about the world and object interactions, making them useful critics during inference to steer robot policies towards success. However, closed-loop robot manipulation requires judging small visual differences between success and failure, which remains a challenge for current VLMs. We introduce a method to fine-tune critics by constructing pairwise progress supervision using success and failure rollouts obtained from a policy. Our fine-tuned critic excels at fine-grained progress reasoning and subtle failure detection, outperforming prior progress reasoning baselines. Additionally, we use an action-conditioned video model to predict the visual effect of several candidate actions sampled from a policy, and show that our critic can correctly identify successful candidates to execute, improving the average policy success rate by 11% across real-world tasks and 5.9% across simulation tasks.

Problem

• vision-language models
• robot manipulation
• fine-grained reasoning
• failure detection
• visual differences

Innovation

• fine-tuned critic
• pairwise progress supervision
• fine-grained reasoning
• action-conditioned video model
• vision-language models