Search papers, labs, and topics across Lattice.
R2-Dreamer, a novel decoder-free Model-Based Reinforcement Learning (MBRL) framework, is introduced to address the challenge of distilling essential information from irrelevant visual details in image-based MBRL. It employs a self-supervised redundancy-reduction objective, inspired by Barlow Twins, as an internal regularizer to prevent representation collapse without relying on data augmentation. Experiments on DeepMind Control Suite and Meta-World demonstrate that R2-Dreamer achieves competitive performance with DreamerV3 and TD-MPC2, while training faster and showing substantial gains on tasks with subtle, task-relevant objects.
Ditch the data augmentation and decoders: R2-Dreamer's Barlow Twins-inspired objective delivers faster, more versatile MBRL, especially when spotting the small stuff matters.
A central challenge in image-based Model-Based Reinforcement Learning (MBRL) is to learn representations that distill essential information from irrelevant visual details. While promising, reconstruction-based methods often waste capacity on large task-irrelevant regions. Decoder-free methods instead learn robust representations by leveraging Data Augmentation (DA), but reliance on such external regularizers limits versatility. We propose R2-Dreamer, a decoder-free MBRL framework with a self-supervised objective that serves as an internal regularizer, preventing representation collapse without resorting to DA. The core of our method is a redundancy-reduction objective inspired by Barlow Twins, which can be easily integrated into existing frameworks. On DeepMind Control Suite and Meta-World, R2-Dreamer is competitive with strong baselines such as DreamerV3 and TD-MPC2 while training 1.59x faster than DreamerV3, and yields substantial gains on DMC-Subtle with tiny task-relevant objects. These results suggest that an effective internal regularizer can enable versatile, high-performance decoder-free MBRL. Code is available at https://github.com/NM512/r2dreamer.
Meta AI