Search papers, labs, and topics across Lattice.
This paper introduces VerNav, a verifier-first framework designed to enhance low-latency vision-and-language navigation (VLN) by substituting per-step autoregressive generation with batched action verification. By employing an entropy-based adaptive generator only for uncertain decisions, VerNav significantly reduces decision-stage latency while maintaining competitive navigation performance. Experiments demonstrate that VerNav achieves over a 10x reduction in average decision-stage latency compared to traditional autoregressive methods on the Room-to-Room benchmark.
VerNav slashes decision-stage latency by over 10x while maintaining competitive navigation performance in vision-and-language tasks.
Vision-and-Language Navigation (VLN) requires an agent to navigate through unseen 3D environments according to natural-language instructions. Explicit reasoning can improve instruction understanding and semantic grounding, but autoregressive generation at every step accumulates large decision-stage latency over multi-step navigation. We propose VerNav, a verifier-first framework for low-latency LLM-based VLN. The verifier reduces decision-stage latency by replacing per-step autoregressive generation with batched action verification, while an entropy-based adaptive generator is invoked only for uncertain decisions to produce compact state evidence. To further improve navigation performance with the verifier, we introduce a two-stage alignment scheme: (i) VPO improves local action-preference alignment in static verifier training, and (ii) step-level reinforcement fine-tuning provides dense progress rewards over multi-step navigation rollouts during dynamic task execution. Experiments on the Room-to-Room (R2R) benchmark show that the verifier-only decision path of VerNav achieves competitive navigation performance among representative LLM-based VLN agents while reducing average decision-stage LLM latency per step by more than $10\times$ compared with autoregressive methods.
