Search papers, labs, and topics across Lattice.
This paper investigates the integration of proprioceptive state in Vision-Language-Action (VLA) models by conducting controlled experiments to assess its impact on closed-loop control across various tasks. The authors explore the effectiveness of different interfaces for incorporating state information, including discrete prompts and feature modulation, while varying state-history length from 1 to 96 frames. Key findings reveal that the choice of interface and the amount of historical state significantly influence model performance, leading to actionable design principles for enhancing state-aware VLA systems.
State history can dramatically enhance VLA model performance, but the interface choice is crucial for optimizing its impact.
Recent Vision-Language-Action (VLA) models almost universally take robot proprioceptive state as input, yet wire it in incompatible ways -- serialized into text prompts, projected into the vision-language prefix, or fed directly to the action expert -- and almost always as a single current frame. Three questions remain open: (1) whether, and on which tasks, current state actually improves closed-loop control; (2) how much state history helps, and whether its benefit reflects genuine temporal variation rather than added conditioning capacity; and (3) where state should enter the model -- the vision-language backbone or the action-generation module. We answer these questions through controlled experiments on a flow-matching VLA, fixing the backbone, training data, action representation, and evaluation protocol throughout. We implement five representative interfaces -- discrete state prompt, VLM prefix, action prefix, state expert, and feature modulation -- under matched implementation details, and evaluate them on 45 atomic tasks spanning three task families plus 20 composite tasks; we then sweep the state-history length from 1 to 96 frames to examine how historical state information affects model performance. The experiments yield systematic answers to all three questions, distilled into testable design principles for state-aware VLAs.
