Search papers, labs, and topics across Lattice.
This paper tackles the challenge of erroneous generalization in goal-conditioned value functions within offline GCRL, particularly in long-horizon tasks. They identify that appropriate inductive bias in the value function is crucial and propose Latent-Aligned Value Learning (LAVL). LAVL integrates latent-representation-based value generalization with hierarchical planning, achieving state-of-the-art performance on OGBench, especially in long-horizon and trajectory stitching datasets.
Overcome the generalization failures plaguing offline goal-conditioned RL, especially in long-horizon tasks, with a new latent representation alignment method that achieves SOTA results.
Offline goal-conditioned reinforcement learning (GCRL) provides a practical framework for obtaining goal-reaching policies from fixed datasets. However, learning a reliable goal-conditioned value function in long-horizon tasks remains challenging. In this paper, we identify erroneous generalization in goal-conditioned value functions as a fundamental bottleneck, and demonstrate that appropriate inductive bias in the value function is crucial for addressing the bottleneck. Building on these findings, we propose Latent-Aligned Value Learning (LAVL), an offline GCRL algorithm that integrates latent-representation-based value generalization with hierarchical planning in a unified framework. Extensive experiments on OGBench demonstrate that LAVL consistently outperforms existing offline GCRL methods, achieving the highest performance on 20 out of 22 datasets. Notably, LAVL exhibits strong performance in long-horizon tasks and trajectory stitching datasets, where prior methods suffer significant performance degradation. Our code is available at https://github.com/oh-lab/LAVL.git.
