Search papers, labs, and topics across Lattice.
This paper reformulates Vision-Language Navigation in Continuous Environments (VLN-CE) as a Hierarchical Markov Decision Process (MDP) to address the inefficiencies of traditional imitation learning and reinforcement learning in closed-loop settings. By abstracting the environment into a topological graph, the authors enable a high-level policy to operate on a macro action space, while a training-free low-level controller manages state transitions, significantly enhancing sample efficiency. The proposed architecture, which includes an action-aware value head for evaluating state values, achieves state-of-the-art performance on the R2R-CE and RxR-CE benchmarks, demonstrating the effectiveness of this hierarchical approach.
Transforming VLN-CE into a hierarchical framework allows agents to navigate unseen environments more efficiently, achieving state-of-the-art results.
Vision-Language Navigation in Continuous Environments (VLN-CE) requires an agent to follow natural language instructions through unseen environments. Existing imitation learning (IL) pipelines struggle in this closed-loop setting: behavior cloning suffers from distribution shift, and DAgger's expert actions become ambiguous upon trajectory deviation. While Reinforcement Learning (RL) offers a natural paradigm to address this, directly applying RL to micro action spaces is sample-inefficient due to reward sparsity. To overcome this bottleneck, we reformulate VLN-CE as a Hierarchical Markov Decision Process (MDP), explicitly decoupling high-level planning from low-level control. By abstracting the environment into a topological graph, our high-level policy operates on a macro action space of frontier nodes, with a training-free low-level controller acting as its state transition, which significantly compresses the decision horizon and makes closed-loop RL tractable. To support RL optimization on the macro MDP, we propose an action-aware value head to effectively evaluate state values under the dynamic frontier action space, powering a graph-based PPO. Extensive experiments demonstrate the effectiveness of our architecture. Finally, our model achieves state-of-the-art performance on the R2R-CE and RxR-CE benchmarks.
