Search papers, labs, and topics across Lattice.
This paper introduces TapSampling, a novel inference-time sampling framework for robotic manipulation that leverages an Action-VAE to generate diverse action candidates from a compressed latent space. A task-progress-understanding verifier, trained on the sequential structure of robotic datasets, then selects the most promising action based on predicted outcomes. Experiments show that TapSampling significantly improves the performance of various generalist policies in both simulated and real-world environments without requiring policy fine-tuning.
Turn your robot's single-shot policy into a robust sampling-based planner at inference time, boosting performance without retraining.
Existing embodied control research demonstrates remarkable performance improvements by scaling training data and model size. We instead explore inference-time strategy as an alternative axis. Non-deterministic generative models, such as diffusion and autoregressive models, have been widely adopted in the field of embodied control. However, the single-shot inference paradigm limits their performance. In this paper, we propose \textbf{TapSampling}, a plug-and-play framework for inference-time sampling. First, we introduce an Action-VAE that represents actions in a low-dimensional latent space by mapping policy-generated initial actions into a compressed posterior distribution, from which any number of latent samples can be drawn and decoded into candidate actions that approximate the true action distribution. Second, we formulate action verification as task-progress outcome prediction, using the intrinsic sequential structure of robotic datasets to train a semantically grounded verifier for interpretable action selection. Furthermore, TapSampling is a policy-agnostic framework. Extensive experiments in both simulated and real-world environments demonstrate that our method substantially improves multiple generalist policies without further policy finetuning. Code and models are available at the project page.
