Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the practical effectiveness of differential privacy (DP) adaptations in large language models (LLMs) by benchmarking privacy risks using advanced attacks like robust membership inference and canary data extraction. The study reveals that the distribution of adaptation data significantly impacts privacy vulnerability, with closer alignment to pretraining data leading to higher risks, even without direct overlaps. Notably, parameter-efficient fine-tuning methods such as LoRA provide superior empirical privacy protection for out-of-distribution data, offering actionable insights for sensitive applications.
Distribution shifts can drastically elevate privacy risks in LLM adaptations, challenging the reliability of theoretical DP guarantees.
Recent work has applied differential privacy (DP) to adapt large language models (LLMs) for sensitive applications, offering theoretical guarantees. However, its practical effectiveness remains unclear, partly due to LLM pretraining, where overlaps and interdependencies with adaptation data can undermine privacy despite DP efforts. To analyze this issue in practice, we investigate privacy risks under DP adaptations in LLMs using state-of-the-art attacks such as robust membership inference and canary data extraction. We benchmark these risks by systematically varying the adaptation data distribution, from exact overlaps with pretraining data, through in-distribution (IID) cases, to entirely out-of-distribution (OOD) examples. Additionally, we evaluate how different adaptation methods and different privacy regimes impact the vulnerability. Our results show that distribution shifts strongly influence privacy vulnerability: the closer the adaptation data is to the pretraining distribution, the higher the practical privacy risk at the same theoretical guarantee, even without direct data overlap. We find that parameter-efficient fine-tuning methods, such as LoRA, achieve the highest empirical privacy protection for OOD data. Our benchmark identifies key factors for achieving practical privacy in DP LLM adaptation, providing actionable insights for deploying customized models in sensitive settings. Looking forward, we propose a structured framework for holistic privacy assessment beyond adaptation privacy, to identify and evaluate risks across the full pretrain-adapt pipeline of LLMs.