Search papers, labs, and topics across Lattice.
The paper introduces BayesNCL, a Bayesian Gated Non-Negative Contrastive Learning method, to improve the interpretability of latent representations learned via contrastive learning. It addresses the "optimization conflict" caused by deterministic similarity measures in standard CL, where common background features are both attracted and repelled, by introducing a probabilistic gating mechanism that filters out task-irrelevant features. Experiments on Imagenet-100 show BayesNCL achieves a 142.1% improvement in semantic consistency compared to SOTA baselines, without sacrificing downstream task performance.
Contrastive learning's entanglement problem may be solved: BayesNCL disentangles representations by selectively filtering task-irrelevant features, leading to a 142% boost in semantic consistency.
While Contrastive Learning (CL) has revolutionized self-supervised representation learning, its latent representations remain highly entangled and opaque, limiting their interpretability in safety-critical applications. We identify that a fundamental cause of this entanglement is the reliance on deterministic similarity measures, which treat all feature dimensions equally. In compositional scenes, this creates an Optimization Conflict: common background features, such as, "blue sky", are encouraged to align in positive pairs but simultaneously repelled in negative pairs, causing gradient oscillations that hinder precise semantic disentanglement. To address this, we propose BayesNCL (Bayesian Gated Non-Negative Contrastive Learning). Unlike standard approaches, BayesNCL introduces a probabilistic gating mechanism that dynamically filters out task-irrelevant, high-frequency common features while selectively retaining discriminative semantics. By formalizing feature selection as a variational inference problem with a sparse Bernoulli prior, our method effectively resolves the optimization conflict. Empirical experimental results on Imagenet-100 demonstrate that BayesNCL achieves a remarkable 142.1% improvement in semantic consistency compared to state-of-the-art baselines, yielding highly interpretable representations without compromising downstream task performance. Code is available at https://github.com/Cui-Peng-624/BayesNCL.
