Track synthesis, materials and catalysis as it is published.
Search papers, labs, and topics across Lattice.
This paper identifies goal grounding, specifically the failure to recognize or estimate the distance to a visible goal, as the primary limitation in current VLM-based object-goal navigation policies. To address this, they introduce StopWatch, a method that explicitly grounds the goal, leading to substantial improvements in success rates and state-of-the-art performance across six benchmarks. Analysis on corrected benchmarks reveals that with effective grounding, the policy's performance approaches that of expert demonstrators, indicating perception, not policy learning, is the remaining bottleneck.
Explicitly grounding the goal object in view unlocks near-expert performance in object-goal navigation, revealing perception rather than policy as the key limitation.
Pretrained VLMs bring rich world knowledge and semantic priors to object-goal navigation, and fine-tuning a VLM into a navigation policy has become a common recipe. Despite this progress, success rates remain low, and it is unclear which capability these fine-tuned policies lack. Our diagnosis identifies goal grounding as the dominant failure. These policies explore well, yet often fail to notice that the goal is already in view or to estimate how far away it is. To address this failure, we ground the goal explicitly. Explicit grounding substantially improves success and reaches state-of-the-art results on six benchmarks across three scene collections. We also correct annotation defects in these benchmarks, so that future methods are evaluated more faithfully. Our analysis on the clean benchmarks shows that, with ground-truth grounding, our policy nearly matches the expert that generated its demonstrations, leaving the headroom in perception rather than policy learning. Our work points to goal grounding as a missing piece of object-goal navigation and offers a simple and effective solution to the task.
BAIR
CMU ML