Search papers, labs, and topics across Lattice.
This paper introduces TAMP-Nav, a novel framework that enhances embodied navigation by aligning large vision-language models (VLMs) with their 2D pre-training through a Pixel-to-3D Action Formulation, which allows VLMs to select 2D pixels for 3D navigation. The framework also incorporates a Selective Reasoning and Anchor-Trajectory Memory mechanism that optimizes memory usage and reasoning processes, retaining only essential information while compressing redundant data. Experimental results show that TAMP-Nav achieves state-of-the-art performance with a success rate of 66.2% on the R2R-CE dataset, demonstrating significant improvements in runtime and sample efficiency with only 90k training trajectories.
TAMP-Nav aligns embodied navigation with VLMs' 2D capabilities, achieving a 66.2% success rate while drastically improving efficiency.
Although Large Vision-Language Models (VLMs) have significantly advanced embodied navigation, their direct deployment remains challenging, as existing methods often force VLMs into unnatural action spaces that misalign with their 2D pre-training priors, compounded by rigid reasoning schedules and inefficient memory management. To overcome these limitations, we propose TAMP-Nav, a unified framework for efficient embodied navigation. First, we introduce a Pixel-to-3D Action Formulation (Point) that reformulates navigation into 2D visual prompting. Specifically, the VLM merely selects 2D pixels, which are then projected into 3D coordinates for a low-level SLAM controller. This design naturally aligns embodied execution with the VLM's inherent 2D visual capabilities. Second, we propose an integrated Selective Reasoning and Anchor-Trajectory Memory mechanism (Think and Memorize), which dynamically triggers Chain-of-Thought and retains high-fidelity memory only at critical nodes, compressing redundant trajectories into lightweight Space-Time Indicators, thereby preserving critical historical information and enhancing spatio-temporal perception. Finally, we design an efficient Two-Level Alignment Paradigm (Align) via Group Relative Policy Optimization (GRPO). By superimposing global outcome rewards with fine-grained process rewards, this dense supervision tightly aligns the agent's cognitive planning with physical environmental feedback, endowing the model with adaptive reasoning capabilities. Experiments demonstrate that TAMP-Nav achieves state-of-the-art performance (e.g., 66.2% SR on R2R-CE) with high runtime and sample efficiency (requiring only 90k training trajectories).
