Search papers, labs, and topics across Lattice.
This paper introduces HALO-WA, a hybrid-attention latent-guided online reinforcement learning framework designed to enhance world-action models for robotic manipulation. By integrating latent features and action priors, HALO-WA enables rapid adaptation to real-world errors, significantly improving task performance in precision manipulation. The framework achieves an impressive success rate increase from 26.4% to 87.1% across four tasks, outperforming the best baseline by 19.2 percentage points with minimal online training time.
HALO-WA boosts robotic manipulation success rates from 26.4% to 87.1% by effectively adapting to real-world errors in just over an hour of training.
World-action (WA) models can generate long-horizon action chunks for general-purpose robotic manipulation, but they remain vulnerable to calibration, perception, and contact-dynamics errors in real-world precision tasks, often failing in the final few millimeters of alignment or insertion. We propose HALO-WA, a hybrid-attention latent-guided online reinforcement learning (RL) framework for WA models, which leverages latent features and action priors from the WA generation process through a lightweight actor-critic adapter to enable fast online adaptation to real deployment errors. HALO-WA introduces a hybrid-attention structure that preserves the temporal consistency of action chunks while reading task-relevant information from WA latents conditioned on visual context and end-stage correction requirements, thereby producing refined action chunks. We validate HALO-WA on four real-world precision manipulation tasks, where it improves the average success rate from 26.4\% for WA-base to 87.1\%, outperforming the strongest baseline by 19.2 percentage points while requiring only 45--75 minutes of online training per task. To facilitate reproducibility, we further conduct supplementary simulation experiments in RoboTwin and release the code at https://github.com/YeanRoot/HALO-WA.
Tsinghua AI