Search papers, labs, and topics across Lattice.
This paper introduces CoJEPA, a novel architecture that integrates Contrastive Learning with Joint-Embedding Predictive Architecture (JEPA) to enhance music representation learning. By jointly training a single backbone with a masked sequence token objective and a contrastive objective, CoJEPA achieves stability and richer representations without the need for an EMA teacher or additional parameters. The results demonstrate that CoJEPA outperforms or matches existing methods across various music information retrieval tasks, particularly excelling in tonal and harmonic understanding.
Combining contrastive learning with JEPA yields a model that enhances music representation without increasing complexity, outperforming traditional methods in key tasks.
Joint-Embedding Predictive Architecture (JEPA) has shown strong performance in learning rich representations through self-supervised prediction in latent space. However, it typically relies on teacher--student architecture with an EMA to stabilise training, and can tend to yield uninformative representations. Contrastive learning is stable to train and produces strong global representations, but remains limited on local tasks by the global nature of its objective. In this work, we combine both into CoJEPA: a single shared backbone jointly trained with a JEPA objective on masked sequence tokens and a contrastive objective on the class token. The contrastive gradient provides stability, removing the need for an EMA teacher entirely, while JEPA enriches the sequence tokens via local predictions that contrastive learning alone cannot provide. Crucially, no extra parameters are added to the backbone: the same model is guided towards richer representations purely through the design of its training signal. CoJEPA takes the best of both worlds, outperforming or matching both individual methods across global and local MIR tasks, with a particularly strong advantage on tonal and harmonic understanding, and without any task-specific architectural changes. CoJEPA shows that combining objectives with complementary inductive biases can substitute for scale, encouraging future work to invest in smarter training objectives over ever-larger models.
