Search papers, labs, and topics across Lattice.
This paper introduces RL^2, an adaptive inference-time steering framework that enhances Vision-Language-Action (VLA) models by leveraging Reinforcement Learning on latent representations. By training a lightweight offline RL policy that conditions on expressive latents, RL^2 effectively combines the behavioral priors of imitation learning with increased action diversity, activating compositional steering only when failure is predicted. The method significantly improves success rates by up to 17.3% in out-of-domain tasks, demonstrating its practical applicability in real-world scenarios.
Action diversity is most valuable when a VLA model is likely to fail, leading to a 17.3% boost in success rates without unnecessary perturbations when success is probable.
Despite the impressive visuomotor capabilities enabled by Vision-Language-Action (VLA) models, their performance often degrades on challenging and out-of-domain tasks. Recent test-time steering and scaling methods improve performance without extensive data collection and retraining, but action samples often remain concentrated around similar behaviors and therefore inherit correlated failure modes. Moreover, existing methods apply the same intervention strategy at every timestep, regardless of whether the base policy is already likely to succeed. To address these limitations, we introduce RL^2, an adaptive inference-time steering framework that leverages Reinforcement Learning on VLA Latents. First, we train a lightweight offline RL policy conditioned on expressive latents extracted from the VLA action expert and compose its flow velocity with that of the frozen VLA during inference. This compositional steering strategy combines the behavioral priors of large-scale imitation learning with the action diversity induced by offline RL beyond dominant demonstration modes. We further discover that inference-time steering follows fundamentally different scaling laws under success and failure states, revealing that action diversity is most beneficial when the base VLA is likely to fail, but can unnecessarily perturb already-accurate actions when success is likely. Building on this insight, RL^2 activates compositional steering only when failure is predicted. Across the SIMPLER and PolaRiS benchmarks, RL^2 improves success rates by up to +17.3% in out-of-domain settings, while ablations and scaling studies demonstrate the importance of latent representations and RL training. Finally, real-world experiments demonstrate that these gains transfer beyond simulation, establishing RL^2 as a practical and modular steering framework for VLA deployment.
NUS