TapSampling: Inference-Time Sampling with a Task-Progress-Understanding Verifier for Robotic Manipulation

2026-05-25

AI Summary

Current single-pass inference paradigms constrain the performance of non-deterministic generative models in robotic manipulation. To address this limitation, this work proposes TapSampling—a plug-and-play, inference-time sampling framework that enables policy-agnostic execution refinement by efficiently exploring the action latent space and incorporating a semantically interpretable task-progress prediction verifier. Built upon an action variational autoencoder (Action-VAE), TapSampling is compatible with both diffusion and autoregressive models and requires no fine-tuning. Experiments demonstrate that it significantly enhances task success rates and robustness across diverse general-purpose policies in both simulated and real-world environments.

Abstract

Existing embodied control research demonstrates remarkable performance improvements by scaling training data and model size. We instead explore inference-time strategy as an alternative axis. Non-deterministic generative models, such as diffusion and autoregressive models, have been widely adopted in the field of embodied control. However, the single-shot inference paradigm limits their performance. In this paper, we propose \textbf{TapSampling}, a plug-and-play framework for inference-time sampling. First, we introduce an Action-VAE that represents actions in a low-dimensional latent space by mapping policy-generated initial actions into a compressed posterior distribution, from which any number of latent samples can be drawn and decoded into candidate actions that approximate the true action distribution. Second, we formulate action verification as task-progress outcome prediction, using the intrinsic sequential structure of robotic datasets to train a semantically grounded verifier for interpretable action selection. Furthermore, TapSampling is a policy-agnostic framework. Extensive experiments in both simulated and real-world environments demonstrate that our method substantially improves multiple generalist policies without further policy finetuning. Code and models are available at the project page.

Problem

• embodied control
• inference-time sampling
• robotic manipulation
• non-deterministic generative models
• action selection

Innovation

• TapSampling
• Action-VAE
• inference-time sampling
• task-progress verifier
• policy-agnostic