Search papers, labs, and topics across Lattice.
This paper introduces T^2VLA, a novel test-time reinforcement learning framework that enables Vision-Language-Action Models (VLAs) to improve their policies without relying on external feedback. By leveraging the internal evaluative capabilities of VLAs, the method uses trajectory-level similarity to high-confidence expert demonstrations as intrinsic rewards, significantly enhancing performance on benchmarks like LIBERO and RoboTwin. The results demonstrate that T^2VLA not only outperforms traditional supervised approaches but also approaches the performance of oracle RL with ground-truth rewards, showcasing its effectiveness across various VLA paradigms.
Trajectories with higher generation confidence in VLAs can drive self-improvement without external rewards, leading to performance on par with oracle RL methods.
Reinforcement learning (RL) has become indispensable for pushing Vision-Language-Action Models (VLAs) beyond static imitation learning. However, existing RL methods typically require external environmental feedback, relying on predefined success signals to guide policy updates. In this work, we show that VLA models possess useful internal evaluative capabilities: in discrete-action VLAs, trajectories with higher generation confidence are significantly more likely to succeed. Based on this observation, we introduce T^2VLA (Test-time VLA), an architecture-agnostic test-time RL framework that enables VLA models to achieve self-bootstrapping policy improvement. Instead of relying on external rewards, T^2VLA leverages trajectory-level similarity to high-confidence expert demonstrations as an intrinsic reward signal. In addition, we propose a Confidence-Driven Dual Expert Bootstrapping mechanism, which dynamically balances a Local Pseudo-Expert for exploration and a Global Expert Pool for training stability. Extensive experiments on the LIBERO and RoboTwin benchmarks show that T^2VLA consistently outperforms supervised baselines and approaches oracle RL performance with ground-truth rewards, achieving effective improvement without external reward feedback. Furthermore, T^2VLA adapts to distinct VLA paradigms, including both OpenVLA-OFT and the pi series.
