Search papers, labs, and topics across Lattice.
This paper introduces a hierarchical approach to offline goal-conditioned reinforcement learning (GCRL) that leverages both temporal and absolute abstraction to improve sample efficiency. The key idea is to learn relativised options and distinct representations at different levels of the hierarchy, enabling the agent to reuse experience across similar state-goal contexts. Experiments demonstrate that this abstraction-based approach significantly improves performance in offline GCRL settings compared to non-hierarchical baselines.
Hierarchical policies aren't just for horizon reduction; they can unlock powerful state-space abstraction for dramatic gains in offline goal-conditioned RL.
Markov Decision Processes (MDPs) often exhibit significant redundancy due to symmetries and shared structure across state-goal pairs in real-world Goal-Conditioned Reinforcement Learning (GCRL). While hierarchical policies have been motivated for horizon reduction via temporal abstraction in offline GCRL, we demonstrate that hierarchy also enables absolute abstraction. By introducing relativised options as well as distinct representations for different levels of the hierarchy, we demonstrate how an agent can reuse experience across similar contexts of the state-space. Based on this framework, we introduce two simple algorithms for learning relativised options and abstracting from the absolute frame of reference. Our experiments show that such inductive biases significantly improve performance in offline GCRL.
