Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Predictive Representation Learning (PRL) as a new category within self-supervised learning, focused on latent prediction of unobserved data components. They propose a taxonomy classifying PRL alongside alignment and reconstruction-based methods, positioning Joint-Embedding Predictive Architecture (JEPA) as a key example of PRL. Empirical comparisons of BYOL, MAE, and I-JEPA reveal that while MAE achieves perfect similarity, I-JEPA and BYOL demonstrate superior robustness, suggesting a trade-off between similarity and robustness in self-supervised learning.
Self-supervised learning can achieve perfect similarity but at the cost of robustness, suggesting that predictive methods like JEPA offer a better trade-off.
Self-supervised learning has emerged as a major technique for the task of learning from unlabeled data, where the current methods mostly revolve around alignment of representations and input recon struction. Although such approaches have demonstrated excellent performance in practice, their scope remains mostly confined to learning from observed data and does not provide much help in terms of a learning structure that is predictive of the data distribution. In this paper, we study some of the recent developments in the realm of self-supervised learning. We define a new category called Predictive Representation Learning (PRL), which revolves around the latent prediction of unobserved components of data based on the observation. We propose a common taxonomy that classifies PRL along with alignment and reconstruction-based learning approaches. Furthermore, we argue that Joint-Embedding Predictive Architecture(JEPA) can be considered as an exemplary member of this new paradigm. We further discuss theoretical perspectives and open challenges, highlighting predictive representation learning as a promising direction for future self-supervised learning research. In this study, we implemented Bootstrap Your Own Latent (BYOL), Masked Autoencoders (MAE), and Image-JEPA (I-JEPA) for comparative analysis. The results indicate that MAE achieves perfect similarity of 1.00, but exhibits relatively weak robustness of 0.55. In contrast, BYOL and I-JEPA attain accuracies of 0.98 and 0.95, with robustness scores of 0.75 and 0.78, respectively.