Search papers, labs, and topics across Lattice.
This paper empirically evaluates a novel visual grounding approach for vision-language-action (VLA) navigation policies, addressing the challenges posed by perceptual distractions and ambiguous scene interpretations. By employing a real-time segmentation method using SegFormer, the authors demonstrate that highlighting traversable areas in green and non-traversable areas in red significantly reduces mean waypoint error by 27-44%, particularly benefiting longer instructions. The findings reveal that while visual grounding enhances navigation efficiency, it primarily serves as a trajectory length regularizer rather than improving reasoning for out-of-distribution instructions.
Visual grounding can cut waypoint error by up to 44% in VLA navigation, especially for longer instructions, without the need for model retraining.
Vision-language-action (VLA) models enable robot navigation from natural language and visual goals, but remain susceptible to perceptual distractions and ambiguous scene interpretations. This paper presents the first empirical evaluation of visual grounding for VLA navigation policies. We propose a real-time segmentation-based grounding method that highlights traversable areas in green and non-traversable areas in red using SegFormer. Two variants are evaluated: observation-only segmentation and joint observation-goal augmentation. Using OmniVLA on the Grand Tour dataset, we show that visual grounding reduces the mean waypoint error by 27-44% at the farthest waypoint, depending on the instruction length. The benefits are greater for long instructions than for short instructions, and grounding provides little improvement for image goals. Normalized error analysis indicates that grounding primarily acts as a trajectory length regularizer, reducing the predicted path length by 30% without improving per-unit-distance reasoning. Our results indicate that visual grounding offers a simple, computationally inexpensive method to improve VLA navigation without model retraining, although it cannot compensate for missing training signals in out-of-distribution instructions.
