Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to representation learning for Reinforcement Learning (RL) by leveraging a temporal correlation space that emphasizes the importance of small, dynamic elements in video data. The proposed Multi-scale Temporal Contrastive Learning (MTCL) method effectively balances attention across different temporal elements, leading to more informative representations that enhance policy learning. Experimental results show that MTCL significantly improves sample efficiency and performance in various downstream RL tasks compared to traditional methods focused on stationary information.
Temporal correlations in video data can unlock a new level of sample efficiency and performance in Reinforcement Learning pre-training.
Unsupervised pre-training on large-scale datasets has demonstrated significant potential for improving the sample efficiency and performance of Reinforcement Learning (RL). Given the large-scale action-free internet videos, existing methods utilize single-step transition prediction and image reconstruction to learn representations. However, these methods prefer to preserve large-proportion stationary information in the pixel space, neglecting small but crucial information. To preserve enough information in the representation, it is essential to pay equal attention to each element in videos. Specifically, we propose a temporal correlation space to distinguish each element. For implementation, we introduce the Multi-scale Temporal Contrastive Learning (MTCL) method to model multi-scale temporal correlations separately. This approach can balance the attention of different elements and yield more informative representations, effectively supporting policy learning in various downstream tasks. Experimental results demonstrate that our method improves sample efficiency and asymptotic performance across various downstream tasks.
