Search papers, labs, and topics across Lattice.
This paper introduces a novel intrinsic exploration method that leverages temporal contrastive learning to guide agents towards states with unpredictable future outcomes. By learning temporal representations that capture information relevant to downstream tasks, the agent actively explores states that enhance its understanding of the environment. The authors demonstrate that this approach enables the learning of complex exploratory behaviors in locomotion, manipulation, and embodied AI tasks, achieving performance comparable to reward-engineered methods.
Forget hand-engineered reward functions: this method learns complex exploratory behaviors by simply predicting which states lead to unpredictable futures.
Effective exploration in reinforcement learning requires not only tracking where an agent has been, but also understanding how the agent perceives and represents the world. To learn powerful representations, an agent should actively explore states that contribute to its knowledge of the environment. Temporal representations can capture the information necessary to solve a wide range of potential tasks while avoiding the computational cost associated with full state reconstruction. In this paper, we propose an exploration method that leverages temporal contrastive representations to guide exploration, prioritizing states with unpredictable future outcomes. We demonstrate that such representations can enable the learning of complex exploratory x in locomotion, manipulation, and embodied-AI tasks, revealing capabilities and behaviors that traditionally require extrinsic rewards. Unlike approaches that rely on explicit distance learning or episodic memory mechanisms (e.g., quasimetric-based methods), our method builds directly on temporal similarities, yielding a simpler yet effective strategy for exploration.
Google Research