Search papers, labs, and topics across Lattice.
This paper introduces Rollout-Retrieval Lifelong Policy Learning (R虏LPL), a framework that enables autonomous driving policies to continually improve by learning from their own mistakes while retaining previously acquired skills. By filtering recoverable mistakes and retrieving corrective targets, R虏LPL transforms sparse failure data into compact supervised knowledge, facilitating stable and efficient policy enhancement. Evaluated on the nuPlan benchmarks, R虏LPL significantly boosts performance, particularly in challenging scenarios, demonstrating its potential for robust continual learning in autonomous driving systems.
Transforming mistakes into actionable knowledge, R虏LPL achieves state-of-the-art performance in autonomous driving with minimal retraining.
Autonomous driving policies should be able to improve continually as deployment exposes them to increasingly diverse and long-tail traffic situations. However, most learning-based policies are trained or fine-tuned on expert demonstrations and then rely largely on generalization to handle challenging closed-loop scenarios, lacking an explicit mechanism to correct and retain the mistakes exposed in these scenarios. This paper studies autonomous driving policy improvement from a lifelong learning perspective: Can a pretrained policy improve continually by accumulating corrective knowledge derived from its own mistakes, while retaining previously acquired driving competence? To answer this question, we propose Rollout-Retrieval Lifelong Policy Learning (R$^2$LPL), a policy learning framework that retrieves corrective targets from recoverable policy-induced mistakes and retains the resulting knowledge through lifelong policy learning. R^2LPL addresses a key bottleneck in continual policy improvement: closed-loop mistakes reveal where the policy is weak, but do not directly specify what the policy should learn. By filtering recoverable mistake-related states and retrieving feasible corrective targets, R$^2$LPL turns sparse failure evidence into compact supervised knowledge for stable and sample-efficient policy improvement. We evaluate R$^2$LPL on large-scale closed-loop nuPlan benchmarks. With only a few rollout and continual-learning cycles, R$^2$LPL elevates a learning-based planner with moderate initial performance to state-of-the-art performance across the evaluated benchmarks, especially on the challenging and long-tail Test14-hard split. These results demonstrate the effectiveness of R$^2$LPL in converting recoverable closed-loop mistakes into corrective knowledge for sustained policy improvement.
