Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
Current joint-embedding self-supervised learning (JEPA) architectures apply anti-collapse objectives after the projection head, leaving the upstream backbone vulnerable to dimensional collapse and low effective rank that degrades transferability. To resolve this representational mismatch, the authors formulate SACReg, a spectral anti-collapse regularizer grounded in $\lambda$-balance theory that regulates the relative scaling of weight matrices across layers to prevent collapse directly within the encoder. Implemented as $\lambda$-JEPA, the method provably enforces balance in linear settings, empirically increases backbone representation rank in deep networks, and outperforms LeJEPA, VISReg, and V-JEPA 2 on ImageNet-1k, eight downstream transfer tasks, Something-Something-v2, and Kinetics-400.
Self-supervised projection heads mask severe dimensional collapse in the upstream backbone, a hidden failure mode that can be eliminated by enforcing spectral weight balance directly across encoder layers.
Joint-embedding self-supervised learning typically combines an invariance objective across augmented views with additional mechanisms to prevent representational collapse. These objectives are often applied after a projection head, while downstream tasks use the backbone representation before the projector. We find that this mismatch does not necessarily prevent dimensional collapse in the backbone, which can retain low effective rank and potentially limit downstream transfer. To address this, we introduce SACReg, a spectral anti-collapse regularizer motivated by an analysis of $λ$-balance, which captures the relative scale of weight matrices across layers. In a two-layer linear network, we show that (i) $λ$-balance prevents collapse, and (ii) our regularizer applied to the backbone induces $λ$-balance. In the nonlinear case, this regularizer leads to anti-collapse as well and, in realistic architectures on ImageNet100, it empirically increases the representations' ranks. We apply SACReg to JEPA and propose $λ$-JEPA, which improves over LeJEPA and VISReg on ImageNet-1k classification and in average linear-probe transfer performance across eight downstream image datasets. On video self-supervised learning, $λ$-JEPA improves over LeVJEPA and V-JEPA 2 on the Something-Something-v2 and Kinetics-400 benchmarks. Code is available at https://github.com/berkerdemirel/lambda-jepa.