Search papers, labs, and topics across Lattice.
This paper addresses the challenge of exploration in 3D environments for curiosity-driven reinforcement learning by introducing spatial persistence and episodic context. They propose using an online 3D reconstruction as a persistent world model, combined with a sequence model over RGB observations to maintain episodic trajectory history. The resulting agent, trained purely via curiosity, demonstrates superior exploration and generalization capabilities compared to RL-based active mapping baselines, and exhibits efficient adaptation to downstream tasks.
Curiosity-driven agents can escape local loops in 3D environments by remembering where they've been and building a persistent map of the world.
Exploration is a prerequisite for learning useful behaviors in sparse-reward, long-horizon tasks, particularly within 3D environments. Curiosity-driven reinforcement learning addresses this via intrinsic rewards derived from the mismatch between the agent's predictive model of the world and reality. However, translating this intrinsic motivation to complex, photorealistic environments remains difficult, as agents can become trapped in local loops and receive fresh rewards for revisiting forgotten states. In this work, we demonstrate that this failure stems from a lack of spatial persistence and episodic context. We show that effective curiosity requires a model of the world that is persistent and continuously updated, paired with an agent that maintains an episodic trajectory history to navigate toward novel regions. We achieve this using an online 3D reconstruction as a persistent model of the world, while the agent policy is parameterized as a sequence model over RGB observations to maintain episodic context. This design enables effective exploration during training while allowing the agent to navigate using solely RGB frames at deployment. Trained purely via curiosity on HM3D, our agent outperforms RL-based active mapping baselines and generalizes zero-shot to Gibson and AI-generated worlds. Our end-to-end policy enables efficient adaptation to downstream tasks, such as apple picking and image-goal navigation, outperforming from-scratch baselines. Please see video results at https://recuriosity.github.io/.
BAIR