Search papers, labs, and topics across Lattice.
This paper introduces MVP-Nav, a novel RGB-only navigation framework that addresses the challenges of zero-shot object goal navigation by aligning perception, planning, and control with real-world 3D environments. By reconstructing explicit physical occupancy from monocular images using 3D foundation models, MVP-Nav creates a global spatial semantic representation that integrates high-level semantic reasoning with low-level physical constraints through a Multi-layer Value Map (MVM). Experimental results show that MVP-Nav surpasses existing depth-free methods, achieving state-of-the-art performance and demonstrating the effectiveness of structured physical priors in navigation tasks without depth sensors.
MVP-Nav achieves state-of-the-art performance in zero-shot object navigation by effectively integrating semantic reasoning with physical constraints, even in the absence of depth information.
Zero-shot Object Goal Navigation (ZSON) with RGB-only perception poses a fundamental challenge for embodied agents, as the absence of explicit depth information introduces severe physical uncertainty and semantic-physical misalignment. Existing approaches either rely on high-level semantic reasoning without geometric grounding or learn end-to-end policies that lack explicit physical constraints, often resulting in semantically plausible but physically unsafe behaviors. In this paper, we propose MVP-Nav, a physical-aware RGB-only navigation framework that aligns perception, planning, and control with the real 3D world. MVP-Nav reconstructs explicit physical occupancy from monocular observations by leveraging 3D foundation models to project 2D semantic instances into 3D oriented bounding boxes, forming a global spatial semantic representation. To unify high-level semantic reasoning and low-level physical constraints, we introduce a Multi-layer Value Map (MVM) that integrates semantic priorities and reconstructed geometry into a shared cost space, enabling physically grounded geometric planning. Extensive experiments on zero-shot object navigation benchmarks demonstrate that MVP-Nav significantly outperforms existing depth-free methods, achieving state-of-the-art performance and validating that structured physical priors can effectively compensate for the absence of active depth sensors.
