Search papers, labs, and topics across Lattice.
This paper explores the efficacy of output watermarking techniques as a solution to training data membership inference challenges in generative models. By leveraging the phenomenon of residual watermark "radioactivity," the authors demonstrate that watermarking can match traditional loss-based methods in membership detection performance, particularly when there is significant subset exposure. The findings suggest that watermarking could serve as a viable alternative for protecting proprietary datasets against membership inference attacks.
Watermarking can rival traditional membership inference methods, achieving comparable detection performance under specific conditions of subset exposure.
A growing body of literature suggests that training data membership inference problems are fundamentally hard tasks in modern language modeling settings. We argue that output watermarking techniques are the right gadget to make training membership tests for generative models more tractable, based on prior results showing that language models exhibit residual watermark "radioactivity" under partially watermarked training datasets. We pit a watermark-based dataset inference approach head-to-head against traditional loss-based membership inference methods and show that watermarking can achieve comparable membership detection performance when subset exposure is high enough, under an alternate set of assumptions.
