Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the unreliability of visual representations in Vision-Language-Action (VLA) models for spatially precise robotic manipulation by introducing AtVLA, a framework that incorporates learnable register tokens into the visual encoder. The study reveals that attention artifacts in vision encoders can negatively impact spatial perception, and by rectifying attention and refining local information, AtVLA significantly enhances localization and action generation. Experimental results show a marked improvement in success rates across various benchmarks, with LIBERO success rates increasing from 94.2% to 98.4% and real-world success from 46.5% to 69.0%.
Spatial perception in VLA models can be dramatically enhanced by integrating learnable tokens that rectify attention artifacts and refine local information.
Visual representations of VLA models remain unreliable for spatially precise robotic manipulation. We uncover that vision encoders in VLAs also exhibit attention artifacts previously documented in generic Vision Transformers, and further show that, in embodied policies, these artifacts are closely associated with spatial perception capabilities acquired during post-training. As the encoder learns task-relevant information such as object location, depth ordering, and local geometry, limited global-token capacity causes part of this information to spill into low-information patch tokens. We introduce AtVLA, a framework that inserts learnable register tokens into the visual encoder. Trained end-to-end using only embodied data and the original action objective, these registers emerge as dedicated carriers of embodied spatial information, while the remaining patch tokens recover clean and spatially faithful attention distributions crucial for precise target localization and fine-grained contact. Clean attention restores reliable localization, but cannot recover geometric details lost in low-resolution observations. AtVLA therefore couples attention rectification with uncertainty-gated local refinement. The action expert samples multiple action chunks and estimates uncertainty from their disagreement; only for uncertain predictions, action-conditioned attention rollout identifies the task-relevant region, which is cropped, re-encoded at high resolution, and appended to the cached prefix for refined action generation. Across LIBERO, SimplerEnv, and a challenging single-view real-world benchmark, AtVLA improves the average LIBERO success rate from 94.2% to 98.4% and real-world success from 46.5% to 69.0%. The cropping is triggered on approximately 30% of replanning steps, resulting in only 1.4-1.6x the total computation of the base model under the representative deployment setting.