Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models

2026-07-04

AI Summary

Frozen vision-language-action (VLA) models struggle to evaluate the long-term consequences of actions and exhibit limited generalization. This work proposes the SVA framework, which decouples Monte Carlo tree search from action evaluation for the first time: it explores the action distribution of a frozen VLA in simulation, collects reward-labeled trajectories, and distills a lightweight Q-value model to assess the long-term outcomes of candidate actions online. Requiring neither fine-tuning nor simulator deployment, SVA enhances decision quality solely through behavioral distillation and uncertainty regularization. Evaluated across multiple embodied tasks, SVA substantially improves success rates on unseen tasks—enabling a 9B-parameter model to outperform a 27B-parameter counterpart by 7 percentage points while reducing inference latency by 27%.

Abstract

Vision-Language-Action (VLA) models acquire broad embodied capabilities through large-scale pretraining, yet their generalization remains far more fragile than that of LLMs and VLMs. The prevailing remedy, post-training via supervised fine-tuning or reinforcement learning, improves task-specific performance but narrows the generalist capability that makes pretraining valuable. We identify a key bottleneck: VLA failures stem not only from action generation but also from action evaluation. A diagnostic pass@k study confirms that frozen VLAs already contain competent behaviors in their output distribution, with overall success rates rising from 33% at pass@1 to 92% at pass@32. Inspired by this, we propose SVA (Search, Value, and Act), a simple framework that equips frozen VLA policies with long-term consequence awareness. SVA first uses Monte-Carlo tree search in simulation to fully explore the VLA's output distribution and collect diverse trajectories annotated with empirical returns; this knowledge is then distilled into a lightweight Q-value model that predicts the expected consequence of candidate actions; at deployment, the frozen VLA proposes multiple candidates and the evaluator selects the one with the highest uncertainty-regularized Q-value, requiring no simulator access. By decoupling action proposal from consequence evaluation, SVA preserves the generalization capacity of the VLA backbone while substantially improving task success rates. Experiments across embodied benchmarks show that SVA consistently improves generalization on unseen tasks and exhibits strong test-time scaling behavior. Strikingly, SVA enables a 9B VLA to outperform a 27B VLA by 7 points at 27% lower inference latency, suggesting that scaling test-time evaluation is more cost-effective than scaling model size.

Problem

• Vision-Language-Action models
• action evaluation
• generalization
• embodied AI
• long-term consequence awareness

Innovation

• Monte-Carlo Tree Search
• Action Evaluation
• Knowledge Distillation
• Test-time Scaling
• Vision-Language-Action Models