Search papers, labs, and topics across Lattice.
This paper introduces AIDA, a novel domain adaptation framework for visual reinforcement learning that effectively addresses sim-to-real transfer challenges under limited target data. By leveraging adaptive imagination to generate reliable semantic rollouts and employing a distribution-shift-aware discriminator, AIDA enhances the learning process without requiring additional interactions with the target environment. Experimental results show that AIDA significantly outperforms existing baselines across multiple tasks, demonstrating its effectiveness in learning meaningful state representations from scarce data.
Adaptive imagination can transform limited target data into reliable rollouts, enabling robust sim-to-real transfer in visual reinforcement learning.
Sim-to-real transfer remains a major obstacle for reinforcement learning (RL), especially for vision-based control where image observations exacerbate the state-distribution shift between simulation and the real world. Domain adaptation (DA) is a promising remedy for this challenge. Prior sim-to-real DA works have demonstrated encouraging results, yet these approaches typically assume substantially more target data, which is not available in practice. Indeed, their performance degrades significantly when the target data budget is reduced. To address this challenge, we propose AIDA (Adaptive Imagination for Domain Adaptation), a domain adaptation framework for visual reinforcement learning that addresses sim-to-real transfer under scarce target data without requiring additional interaction with the target environment. Our key idea is adaptive imagination: generating reliable and semantic imagination rollouts to augment limited target data. Specifically, AIDA employs a distribution-shift-aware discriminator that truncates rollouts when imagined transitions drift into low-confidence regions, so that only reliable transitions contribute to the augmentation. On these reliable transitions, AIDA introduces a self-consistency loss that cycles through state -> image observation -> state, penalizing discrepancies between the original and reconstructed states. This provides additional adaptation signals beyond the scarce target data. Our experiments demonstrate that adaptive imagination effectively truncates unreliable rollouts. By enforcing a self-consistency loss on the resulting reliable transitions, AIDA learns semantically meaningful state representations and outperforms baselines across five MuJoCo tasks and two Gymnasium-Robotics tasks.
