Search papers, labs, and topics across Lattice.
This paper introduces LMP2, a browser-based tool for individuals to self-audit LLM-generated associations about them, and reports findings from two user studies (N=458) using GPT-4o. The studies reveal that GPT-4o can predict personal attributes with reasonable accuracy and that users desire control over these associations, even when they don't perceive them as privacy violations. The authors also validate their probing method by evaluating eight LLMs on public figures and non-existent names, highlighting the challenges in defining and validating model-individual associations in generative AI.
LLMs can predict surprisingly accurate personal information about individuals, but people struggle to define what constitutes a privacy violation in AI-generated content, revealing a critical gap in human-centered privacy audits.
Large language models (LLMs) learn statistical associations from massive training corpora and user interactions, and deployed systems can surface or infer information about individuals. Yet people lack practical ways to inspect what a model associates with their name. We report interim findings from an ongoing study and introduce LMP2, a browser-based self-audit tool. In two user studies ($N_{total}{=}458$), GPT-4o predicts 11 of 50 features for everyday people with $\ge$60\% accuracy, and participants report wanting control over LLM-generated associations despite not considering all outputs privacy violations. To validate our probing method, we evaluate eight LLMs on public figures and non-existent names, observing clear separation between stable name-conditioned associations and model defaults. Our findings also contribute to exposing a broader generative AI evaluation crisis: when outputs are probabilistic, context-dependent, and user-mediated through elicitation, what model--individual associations even include is under-specified and operationalisation relies on crafting probes and metrics that are hard to validate or compare. To move towards reliable, actionable human-centred LLM privacy audits, we identify nine frictions that emerged in our study and offer recommendations for future work and the design of human-centred LLM privacy audits.
