Search papers, labs, and topics across Lattice.
This paper investigates the limitations of existing privacy measures in user-LLM interactions by introducing a probabilistic variant of Privacy-Conscious Delegation (PCD) that accounts for both explicit identifiers and quasi-identifying traits. By creating the PUPA-SD dataset, which captures naturalistic user queries with self-disclosure, the authors enhance PCD with a k-anonymity estimation approach that optimizes privacy without sacrificing utility. The results show that the proposed method, PAPILLON, significantly improves the privacy-utility balance for larger models like Llama-3.2-3B, while smaller models struggle to achieve similar outcomes.
Optimizing privacy in LLM interactions reveals that larger models can achieve a superior balance of privacy and utility, while smaller models falter.
Recent work on protecting privacy during user-LLM interactions often focuses on direct, explicit identifiers: the personally-identifiable information (PII) captured by standard detectors. One such approach is Privacy-Conscious Delegation (PCD), where a local LLM acts as an intermediary. However, privacy risk does not stem solely from explicit identifiers but also PII-free self-disclosures, leaving users identifiable through combinations of quasi-identifying traits. We investigate a probabilistic variant of PCD, where we augment its objectives with an LLM-driven probabilistic estimation of k-anonymity. To facilitate this, we first create the PUPA-SD dataset, which contains naturalistic user queries with self-disclosure. Our preliminary results indicate that optimizing PAPILLON on PUPA-SD improves quality on unseen conversations across a variety of local models and produces the best privacy-utility balance for Llama-3.2-3B, while smaller models struggle to jointly optimize quality and privacy. We propose k-anonymity as a useful auxiliary metric for tackling PCD.
