Search papers, labs, and topics across Lattice.
This paper introduces VLN-AVP, a novel zero-shot navigation framework for Autonomous Valet Parking (AVP) that integrates Vision-Language Models (VLMs) with a hybrid long-short-term memory system. By eliminating the reliance on pre-built maps and enabling navigation through natural language instructions, the framework interprets complex parking environments more intuitively. Experimental results show that VLN-AVP significantly outperforms existing methods, achieving over a 25% improvement in success rates in simulation and leading performance in real-world tests.
VLN-AVP achieves over 25% higher success rates in autonomous valet parking by leveraging zero-shot navigation and hybrid memory systems, redefining scalability in unseen environments.
Existing methods in Autonomous Valet Parking (AVP) typically rely on pre-built maps, which severely restricts their scalability to unseen environments and open-vocabulary targets. Inspired by the application of Vision-Language Models (VLMs) in Vision-Language Navigation (VLN) tasks, we propose VLN-AVP, a zero-shot navigation framework for AVP tasks. By combining the precise spatial perception of a Bird's-Eye-View (BEV) model with the general intelligence of VLMs, our framework 1) eliminates the dependency on pre-built maps, 2) interprets semantic environmental contexts in parking scenarios, and 3) enables intuitive navigation following natural language instructions. Specifically, we introduce a hybrid memory system: a short-term perception memory tracks semantic visual cues to address the limitations of VLM's single-frame reasoning in existing methods, while a long-term topological memory facilitates stable policy learning from past experiences. To bridge the gap in existing benchmarks, we also present the VLN-AVP dataset and benchmark. Featuring 10 high-fidelity parking scenes and over 1,000 navigation episodes, it has the largest number of garage scenes to date and is the first VLN benchmark for underground parking. Extensive experiments demonstrate that in simulation, our method achieves an over 25% improvement in success rate compared to VLN methods and an over 15% improvement compared to other autonomous driving methods. Furthermore, it attains a leading success rate in real-world vehicle experiments, proving its practical feasibility.
