Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a Poisoned Data Detector (PDD) that enhances the integrity of self-supervised learning (SSL) curated datasets, addressing the risks of data poisoning from unvetted external sources. By combining the pretrained ImageBind model with traditional classifiers like SVM, Random Forest, and k-Nearest Neighbors, the PDD effectively identifies and mitigates adversarial attacks on datasets. The evaluation on 176,200 images reveals that the SVM-PDD outperforms other methods in both in-distribution and out-of-distribution scenarios, showcasing its robustness and scalability for foundation model training.
SVM-PDD outperforms traditional classifiers in detecting poisoned data, ensuring the integrity of self-supervised datasets crucial for robust foundation models.
Self-supervised data curation provides a pathway to scaling and improving the generalization capabilities of machine learning models. By leveraging self-supervised learning (SSL) for data curation, the demand for massive training datasets required by foundation models can be effectively met. SSL greatly alleviates the costs associated with annotation and manual dataset curation while minimizing the need for human oversight. However, the integrity of SSL-curated datasets must be rigorously checked, as reliance on anonymous and unvetted external sources can substantially increase the risk of data poisoning. In this paper, we propose a Poisoned Data Detector (PDD), an active defense mechanism designed to ensure the integrity of SSL-curated datasets prior to foundation model training. PDDs are designed using a combination of the pretrained ImageBind model and traditional classifiers, including Random Forest (RF), k-Nearest Neighbors (KNN), Naive Bayes (NB), and Support Vector Machines (SVM). We rigorously evaluated PDDs using 176,200 images from three diverse datasets and three different adversarial attacks encompassing both in-distribution and out-of-distribution scenarios. Notably, SVM-PDD achieves superior performance for both in-distribution (Set3-Set5) and out-of-distribution (TrueFace and 140K RealFace) datasets. Our design demonstrates strong scalability and enables the rapid integration of new adversarial attack detectors through an ensemble approach.