Search papers, labs, and topics across Lattice.
This critique audits the experimental methodology of *Alignment Whack-a-Mole*, which claimed that fine-tuning enables adversaries to extract substantial portions of copyrighted books from language models. The analysis reveals that the original findings relied on sequence match lengths far shorter than community memorization standards, risked target text leakage through the prompting setup, and omitted essential negative controls to bound false-positive matches. Consequently, the original paper's empirical evidence fails to substantiate claims of functional book extraction or practical copyright infringement.
Headline claims that fine-tuning effortlessly extracts copyrighted books collapse under scrutiny, driven by sub-standard match thresholds, prompt leakage, and zero negative controls.
After careful review, I'm confident the headline fine-tuning memorization results in Alignment Whack-a-Mole use an invalid measurement procedure. The book memorization coverage metric these headline results depend on counts sequence matches far shorter than what field standards consider valid evidence of memorization, and the prompting procedure used to elicit memorization runs the risk of leaking the text being"extracted"in the prompt. The paper doesn't include the negative-control experiments needed to see how much the results are inflated by false positives: claiming extraction success (and therefore memorization of training data) when matches between generations and training data may be due to other factors. Given these validity issues, the paper's claims that fine-tuning lets users extract substantial portions of copyrighted books, in a form that could substitute for the originals, aren't supported by the reported results. The failure to report the experiments'cost (an important component of the threat model) further compromises the copyright claims. I'm writing this note because, in the last month, (prospective) plaintiffs have reached out to me to ask about this paper. They're looking to cite this work as valid evidence in support of claims in ongoing and potential future copyright litigation.
