Search papers, labs, and topics across Lattice.
This study investigates the role of recurrence in vision-language-action (VLA) models, specifically isolating its effects by augmenting a pretrained transformer with learnable memory tokens updated through self-attention. The results show that the $μ$VLA model significantly enhances performance in partially observable environments, achieving a success rate improvement from 0.42 to 0.84 across multiple tasks. Notably, while $μ$VLA excels in scenarios with consistent memory structures, it does not regress in fully observable tasks, clarifying the conditions under which minimal recurrence is beneficial.
Isolating recurrence in VLA models reveals it can double success rates in partially observable tasks without complicating architecture or introducing auxiliary losses.
Vision-language-action (VLA) models predict chunks of future actions from the current observation, an assumption that fails under partial observability, where decisions depend on information no longer visible. Existing memory-augmented VLAs simultaneously introduce recurrence, retrieval, compression modules, auxiliary objectives, hierarchical memory, or task-specific architectural changes, so the contribution of recurrence itself remains entangled with surrounding machinery. We present a controlled isolation study of recurrence in a strong pretrained VLA backbone. Our formulation augments the transformer with a small set of learnable memory tokens carried across timesteps and updated through self-attention, trained end to end with truncated backpropagation through time, with no auxiliary losses and no architectural changes. We instantiate this as $μ$VLA, a family of OpenVLA-OFT variants parameterized by memory width m, TBPTT length K, and the memory update rule (cross-step gradients or a detached EMA), so that recurrence is the only varying factor. On MIKASA-Robo, $μ$VLA improves average success rate on five training tasks from 0.42 to 0.84 at the strongest setting and reaches 0.23 on held-out tasks with the same memory structure versus 0.07 for the memoryless baseline. On tasks requiring different memory structure, performance remains near baseline. On LIBERO, the strongest recurrent variant achieves 96.2% average success, indicating no regression under full observability. We interpret these results as a calibration of the capability envelope of minimal in-backbone recurrence, identifying the regime in which it is sufficient and the regime where additional memory structure is required. Demos and videos can be found in https://avanturist322.github.io/mu-vla/.
