Search papers, labs, and topics across Lattice.
This paper introduces the Distribution Provenance Audit (DPA), a novel framework for post-hoc auditing of data provenance in LLM fine-tuning that addresses the limitations of existing techniques, which are often fragile and require preemptive measures. By leveraging intrinsic distributional fingerprints that capture the lexical-semantic intersection of fine-tuned models, DPA formulates the audit as a statistical hypothesis test, effectively identifying unauthorized data usage even in the presence of adversarial obfuscation tactics. Experimental results demonstrate that DPA significantly outperforms current baselines in medical and legal contexts, while also revealing a dual-use tension where these fingerprints could be exploited for privacy attacks.
Auditing LLM fine-tuning for data IP infringement is now possible even against sophisticated adversarial tactics, thanks to a new framework that quantifies intrinsic distributional fingerprints.
The proliferation of customized Large Language Models (LLMs) poses critical risks of Data Intellectual Property (Data IP) infringement via unauthorized fine-tuning on proprietary data. Existing audit techniques are limited, as they require intervention during data preparation or training and remain fragile under malicious obfuscations such as data paraphrasing and knowledge distillation. We propose \textit{Distribution Provenance Audit (DPA)}, a post-hoc framework for auditing data IP infringement in LLM fine-tuning under black-box and malicious settings. DPA is grounded in a critical insight: regardless of fine-tuning tactics to evade provenance, the practical necessity of maintaining utility constrains the model to preserve the fundamental intersection of semantic substance and lexical form. Accordingly, DPA captures this persistent lexical-semantic intersection as intrinsic distributional fingerprints. The framework formulates the audit as a statistical hypothesis test, effectively quantifying these fingerprints via unbiased output sampling to reliably reject the null hypothesis of non-usage. Extensive experiments on medical and legal fine-tuning tasks show that DPA consistently outperforms existing baselines, remaining robust against adversarial trainers employing paraphrasing and knowledge distillation. We further highlight a fundamental dual-use tension: the same high-fidelity distributional fingerprints enabling reliable auditing may also facilitate privacy attacks.
