Search papers, labs, and topics across Lattice.
This paper introduces the Platonic Representation Defense, a novel black-box test-time defense mechanism for self-supervised learning models against backdoor attacks. By leveraging a conditional energy function based on the Platonic Representation Hypothesis, the method enables detection and purification of representations without requiring access to labels or training data. Experimental results demonstrate significant performance improvements across various self-supervised encoders and over 10 different attack scenarios, highlighting its robustness and versatility.
Backdoor attacks on self-supervised models can be effectively countered without any reliance on labels or training data, achieving substantial performance gains across multiple attack types.
Self-supervised learning (SSL) pretrained models have become a dominant paradigm for visual representation learning, but they are vulnerable to backdoor attacks. Existing defenses struggle to defend against such attacks in a fully black-box setting because they often require access to labels, attack patterns, or training data. To tackle this issue, we propose a new attack-agnostic, model-agnostic, and modality-agnostic black-box test-time defense paradigm, called \emph{Platonic Representation Defense}. It is inspired by the Platonic Representation Hypothesis, which suggests that large-scale independently trained encoders converge toward compatible projections of the same underlying reality. We formalize this idea as a conditional energy function defined over source representations and a set of reference representations. The energy function is trained for detection through noise-contrastive estimation and for representation purification through denoising score matching. Theoretically, the energy gap between matched and mismatched samples is lower bounded by the mutual information between source and reference representations. We demonstrate the effectiveness of our method on multiple self-supervised encoders and more than 10 attacks. The method can perform both representation detection and purification, and achieves substantial performance gains across multiple attacks. Code is available \href{https://github.com/jsrdcht/Platonic-Representation-Defense}{here}.
