Search papers, labs, and topics across Lattice.
This paper introduces DP-GCL, a differentially private contrastive learning framework that reduces inter-sample dependency by partitioning batches into disjoint groups and restricting negative samples to within-group samples. To compensate for the reduced negative sample diversity, they introduce intra-group augmentation. Experiments on eight datasets show that DP-GCL significantly improves performance over existing DP contrastive learning methods, achieving gains of 5.6% in image classification and 20.1% in image-text retrieval.
Differentially private contrastive learning no longer needs to sacrifice so much accuracy, thanks to a new method that cleverly bounds gradient dependencies.
Differentially private (DP) contrastive learning aims to learn general-purpose representations from sensitive data, alleviating the privacy leakage concerns of organizations deploying or sharing embedding models trained on private user content. However, existing approaches suffer from severe utility degradation due to the over-strong inter-sample dependency inherent in standard contrastive objectives, where each sample's gradient depends on all other samples in the batch, amplifying the impact of DP noise. In this work, we argue that effective DP contrastive learning requires explicitly reducing such intrinsic inter-sample reliance. To this end, we propose DP-GCL, a principled DP contrastive learning framework that structurally limits gradient dependency through bounding group-level contribution. DP-GCL partitions each batch into small, disjoint groups and restricts available negative samples to within-group samples, thereby localizing gradient influence and reducing sensitivity. To counteract the resulting loss of negative sample diversity, we further introduce intra-group augmentation, which generates additional negative views without increasing privacy cost. Extensive experiments across eight datasets demonstrate that DP-GCL consistently advances the state of the art in both uni-modal and multi-modal contrastive learning under practical privacy budgets: it improves image classification accuracy by 5.6% and image-text retrieval accuracy by 20.1% over existing DP contrastive methods.
