Search papers, labs, and topics across Lattice.
This paper introduces GenDa, a unified framework for improving data efficiency and generalization in unsupervised reinforcement learning (URL) by addressing the challenges of non-stationary skill semantics and brittle generalization. The authors implement a skill relabeling mechanism to enhance data efficiency during pre-training and a Complementary Information Bottleneck (CIB) to ensure robustness against distribution shifts in downstream tasks. Experimental results show that GenDa outperforms existing URL methods, demonstrating significant improvements in scalability, generalizability, and data efficiency.
GenDa redefines unsupervised reinforcement learning by achieving superior generalization and data efficiency through innovative skill relabeling and robust feature focus.
Unsupervised Reinforcement Learning (URL) aims to pre-train scalable, skill-conditioned policies without extrinsic rewards, serving as a foundation for downstream control tasks. Despite recent progress, we argue that current off-policy URL methods are limited by two critical, overlooked bottlenecks: (1) non-stationary skill semantics and (2) brittle generalization. To address these challenges, we propose GenDa (Generalizable Data-efficient Agent), a unified framework for robust unsupervised reinforcement learning. First, we introduce a skill relabeling mechanism to mitigate non-stationarity and significantly improve data efficiency for pre-training. Second, we propose a Complementary Information Bottleneck (CIB), encouraging the learned skill policy to focus on ego-centric features and become robust to distribution shifts for downstream tasks. Through various experiments, we demonstrate that GenDa significantly enhances the scalability of URL with superior generalizability and data efficiency. Our code and videos are available at https://ihatebroccoli.github.io/official-GenDa.
