Search papers, labs, and topics across Lattice.
This paper introduces Mind-VLA, an instruction-aware spatial representation alignment method designed to enhance Vision-Language-Action (VLA) models by focusing on the specific target object indicated by language instructions. By extracting features from a tri-view of the target object and aligning the VLA model's latent representation with these features, Mind-VLA significantly improves performance on tasks requiring fine-grained manipulation and understanding of occluded targets. The method achieves a remarkable 93.9% accuracy on the LIBERO benchmark and a 54% success rate in real-robot tasks, surpassing previous instruction-agnostic approaches by a substantial margin.
Mind-VLA outperforms traditional instruction-agnostic methods by 32 percentage points in real-robot tasks, showcasing the power of instruction-aware spatial alignment.
Recent Vision-Language-Action (VLA) methods improve generalization by aligning their representations with 3D scene geometry. However, these methods are fundamentally instruction-agnostic: the representations align the entire scene uniformly, neglecting the 3D geometry of the specific target object designated by the language instruction. This causes failures on fine-grained manipulation and target occlusion tasks, where success depends on accurate 3D understanding of the target object rather than the entire scene. To address this, we present Mind-VLA, an instruction-aware spatial representation alignment method for VLA models. Specifically, Mind-VLA first obtains the target object specified by the language instruction, then prepares its target-object tri-view and extracts the corresponding VAE and VGGT features. Finally, the latent representation of the VLA model is aligned with these features to enable instruction-aware 3D understanding. Mind-VLA reaches 93.9% on LIBERO and 4.47 on CALVIN with a compact 345M-parameter backbone. On real-robot tasks with target occlusion, Mind-VLA reaches 54% average success, outperforming the best-performing instruction-agnostic method in real-robot comparison by 32 percentage points. Code will be publicly available.
