Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Incremental Residual Reinforcement Learning (IRRL) for efficient social navigation in real-world robotic applications. IRRL combines incremental learning, which avoids replay buffers and batch updates, with residual RL, which focuses learning on the difference between a base policy and the optimal policy. Experiments show that IRRL achieves comparable performance to replay buffer-based methods in simulation and enables effective adaptation to new environments in real-world robot navigation.
Robots can now adapt to new social navigation environments in the real world without the computational overhead of replay buffers, thanks to a clever combination of incremental and residual RL.
As the demand for mobile robots continues to increase, social navigation has emerged as a critical task, driving active research into deep reinforcement learning (RL) approaches. However, because pedestrian dynamics and social conventions vary widely across different regions, simulations cannot easily encompass all possible real-world scenarios. Real-world RL, in which agents learn while operating directly in physical environments, presents a promising solution to this issue. Nevertheless, this approach faces significant challenges, particularly regarding constrained computational resources on edge devices and learning efficiency. In this study, we propose incremental residual RL (IRRL). This method integrates incremental learning, which is a lightweight process that operates without a replay buffer or batch updates, with residual RL, which enhances learning efficiency by training only on the residuals relative to a base policy. Through the simulation experiments, we demonstrated that, despite lacking a replay buffer, IRRL achieved performance comparable to those of conventional replay buffer-based methods and outperformed existing incremental learning approaches. Furthermore, the real-world experiments confirmed that IRRL can enable robots to effectively adapt to previously unseen environments through the real-world learning.