Search papers, labs, and topics across Lattice.
This paper introduces JOP-VLN, a novel framework for Vision-and-Language Navigation that integrates off-policy imitation learning and on-policy exploration within a three-stage training pipeline. The approach begins with imitation learning from expert demonstrations, followed by the DAgger algorithm to enhance error recovery, and culminates in a joint learning framework that employs high-entropy trajectory sampling for efficient reinforcement learning. Extensive experiments on the VLN-CE R2R and RxR benchmarks demonstrate that JOP-VLN achieves state-of-the-art success rates of 69.9% and 68.0%, respectively, highlighting its effectiveness in navigating complex environments using natural language instructions.
JOP-VLN achieves a groundbreaking 69.9% success rate in Vision-and-Language Navigation by seamlessly integrating imitation learning with reinforcement learning for enhanced error recovery and exploration.
Vision-and-Language Navigation (VLN) necessitates an embodied agent to navigate in the physical world by adhering to natural language instructions. Recent advancements in Vision-Language Models (VLM) have propelled the development of VLM-based VLN methods with two predominant paradigms: (1) imitation learning (IL) on expert demonstrations, followed by the Dataset Aggregation (DAgger) algorithm to bolster error recovery capabilities; (2) reinforcement learning (RL) driven by verifiable rewards to enhance reasoning and exploration. A notable gap is the absence of integration between these two distinct paradigms. This paper introduces JOP-VLN, a novel VLN framework that synergistically combines off-policy imitation learning and on-policy exploration within a three-stage training pipeline. Initially, IL is employed on expert demonstrations to acquire basic navigation skills. Subsequently, the DAgger algorithm is utilized to generate heuristic exploration trajectories, which are then used for imitation learning to improve error recovery capabilities. Finally, a joint on-and-off policy learning framework is implemented, featuring high-entropy trajectory sampling to enhance RL training efficiency and an error-correction-prioritized trajectory sorting strategy for effective error correction. Extensive experiments demonstrate the efficacy of JOP-VLN, achieving success rates of 69.9% and 68.0% on the VLN-CE R2R and RxR benchmarks, respectively, setting a new state-of-the-art on R2R. Project page: https://qingrongh.github.io/JOP-VLN.
