Search papers, labs, and topics across Lattice.
This paper introduces an online continual reinforcement learning framework for robotic agents that leverages DreamerV3 and world model prediction residuals to detect out-of-distribution events and trigger finetuning. Adaptation progress is monitored using task-level performance and internal training metrics, enabling convergence assessment without external supervision. The approach is validated on continuous control problems, including a quadruped robot in simulation and a real-world model vehicle, demonstrating the potential for autonomous adaptation in robotic systems.
Robots can now adapt to unforeseen circumstances during deployment by leveraging world model prediction errors to trigger online continual RL, moving beyond static training regimes.
As learning-based robotic controllers are typically trained offline and deployed with fixed parameters, their ability to cope with unforeseen changes during operation is limited. Biologically inspired, this work presents a framework for online Continual Reinforcement Learning that enables automated adaptation during deployment. Building on DreamerV3, a model-based Reinforcement Learning algorithm, the proposed method leverages world model prediction residuals to detect out-of-distribution events and automatically trigger finetuning. Adaptation progress is monitored using both task-level performance signals and internal training metrics, allowing convergence to be assessed without external supervision and domain knowledge. The approach is validated on a variety of contemporary continuous control problems, including a quadruped robot in high-fidelity simulation, and a real-world model vehicle. Relevant metrics and their interpretation are presented and discussed, as well as resulting trade-offs described. The results sketch out how autonomous robotic agents could once move beyond static training regimes toward adaptive systems capable of self-reflection and -improvement during operation, just like their biological counterparts.
