Search papers, labs, and topics across Lattice.
This paper investigates the impact of stochastic resetting on reinforcement learning, where the agent is intermittently returned to a fixed state. They find that resetting accelerates policy convergence in tabular grid environments, even when it doesn't improve search time for a diffusive agent, suggesting a novel mechanism beyond first-passage optimization. Furthermore, they demonstrate that stochastic resetting enhances deep reinforcement learning in continuous control tasks with sparse rewards by truncating uninformative trajectories and improving value propagation.
Stochastic resetting鈥攔andomly teleporting RL agents back to the start鈥攕urprisingly speeds up learning, even when it wouldn't help a non-learning agent.
Stochastic resetting, where a dynamical process is intermittently returned to a fixed reference state, has emerged as a powerful mechanism for optimizing first-passage properties. Existing theory largely treats static, non-learning processes. Here we ask how stochastic resetting interacts with reinforcement learning, where the underlying dynamics adapt through experience. In tabular grid environments, we find that resetting accelerates policy convergence even when it does not reduce the search time of a purely diffusive agent, indicating a novel mechanism beyond classical first-passage optimization. In a continuous control task with neural-network-based value approximation, we show that random resetting improves deep reinforcement learning when exploration is difficult and rewards are sparse. Unlike temporal discounting, resetting preserves the optimal policy while accelerating convergence by truncating long, uninformative trajectories to enhance value propagation. Our results establish stochastic resetting as a simple, tunable mechanism for accelerating learning, translating a canonical phenomenon of statistical mechanics into an optimization principle for reinforcement learning.
Stanford HAI