Search papers, labs, and topics across Lattice.
FutureNav introduces a unified world-action modeling framework for vision-and-language navigation (VLN) that integrates text, visual, and spatial features into a single large language model (LLM). By optimizing for action prediction, state transitions, and future spatial state generation, FutureNav enhances navigation performance while maintaining inference speed. Experimental results demonstrate that FutureNav, utilizing a 4B-scale backbone, achieves state-of-the-art results across multiple VLN benchmarks, significantly outperforming previous methods.
FutureNav redefines VLN by simultaneously predicting actions and modeling world states, achieving unprecedented performance with a streamlined architecture.
Vision-and-language navigation (VLN) in continuous environments requires an agent to ground instructions in egocentric observations while maintaining spatial understanding across long action sequences. Recent navigation foundation models have shown strong progress by scaling vision-language models, but they often learn navigation primarily as direct action generation, without explicitly modeling world states or predicting their future evolution. We introduce FutureNav, a VLM-based unified world-action modeling framework for vision-and-language navigation. Specifically, FutureNav jointly encodes text, visual, and spatial features and feeds them into the LLM, and optimizes four objectives for simultaneous world and action modeling: an action policy objective for navigation action prediction, inverse and forward dynamics objectives for modeling state transitions, and a future generation objective for predicting future spatial states. This unified architecture strengthens action prediction while explicitly modeling the world, without sacrificing inference speed. Extensive experiments show that, with only a 4B-scale backbone, FutureNav achieves state-of-the-art performance on multiple VLN benchmarks and substantially outperforms prior VLN methods, paving the way toward future world-action models for VLN. We will release the code and models to support future research.
