Search papers, labs, and topics across Lattice.
This paper introduces the X-NavDP framework, which enhances navigation diffusion policies by employing a data-efficient reinforcement learning (RL) post-training approach called Group Q-score Reweighted Matching (GQRM). The method addresses the limitations of traditional RL in diffusion settings by implementing a self-bootstrapped exploration strategy and a group Q-score normalization mechanism, enabling robust policy updates across diverse robotic embodiments. As a result, X-NavDP significantly improves navigation success rates in both simulated environments (from 61.20% to 84.28%) and challenging real-world scenarios (from 10% to 65%).
Achieving a remarkable 23% increase in navigation success rates across diverse robotic embodiments, X-NavDP redefines the potential of diffusion policies in complex environments.
Pretraining navigation diffusion policies rely on large-scale expert demonstrations. These data are typically generated by a fully-informed oracle planner suited to a single nominal robot. This limits the policy's generalization to diverse embodiments and challenging scenarios (e.g., escaping dead ends or detouring long obstacles) that demand diverse local reactive behaviors with only onboard local observations. Post-training the policy with reinforcement learning (RL) offers a principled remedy. However, previous RL for diffusion approaches lead to only marginal improvements. This is because the intractable likelihood of diffusion policies renders policy gradients unstable in addition to inefficient policy exploration. To address these challenges, we propose a data-efficient diffusion RL post-training framework - GQRM (Group Q-score Reweighted Matching). Our framework introduces two complementary designs: (i) a self-bootstrapped exploration strategy with behavior perturbation that preserves the pretrained policy prior, and (ii) a group Q-score normalization mechanism that computes per-trajectory values on each state for efficient reweighted score matching. By conducting distributed online RL training across heterogeneous embodiments, the resulting fine-tuned policy, X-NavDP, achieves state-of-the-art cross-embodiment visual navigation performance, improving the overall success rate from 61.20% to 84.28% in simulation and 10% to 65% in real-world hard cases. The code and model are publicly available at https://yty-sky.github.io/x-navdp-project-page.
Tsinghua AI