Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces PSALM, an evaluation framework that operationalizes EU copyright doctrine to assess stylistic appropriation by large language models (LLMs). By applying PSALM to Llama 3.2 models fine-tuned on Dutch literary works, the authors reveal that instruction-tuned models exhibit significant baseline stylistic similarity and that fine-tuning leads to systematic stylistic appropriation across various dimensions, not just verbatim memorization. The study highlights that existing safeguards are inadequate for addressing broader copyright risks, emphasizing the need for a more comprehensive compliance evaluation framework.
Fine-tuning LLMs can lead to systematic stylistic appropriation that goes beyond mere verbatim copying, raising serious copyright concerns under EU law.
Large language models (LLM) trained on web-scale corpora generate output that may infringe copyright, yet existing technical safeguards focus narrowly on verbatim memorisation. EU copyright doctrine applies a broader standards: substantial similarity, which extends to stylistic choices, narrative structure, and creative elaboration. This mismatch between what current methods detect and what the law protects leaves a significant compliance gap. We introduce PSALM, an LLM-as-a-judge framework that operationalises EU copyright doctrine through ten evaluators assessing computational overlap, stylistic dimensions (writing style, narrative voice), content dimensions (character, plot, scene, world building), and statutory exceptions (parody, pastiche, quotation, scènes à faire). Applying PSALM to Llama~3.2 models fine-tuned on translated historical Dutch literary works, we find that: 1) instruction-tuned models exhibit non-trivial baseline stylistic similarity prior to corpus exposure; 2) fine-tuning induces systematic stylistic appropriation across all infringement-relevant dimensions, extending beyond verbatim memorisation to abstract narrative patterns; 3) Negative Preference Optimisation unlearning substantially reduces similarity but leaves detectable residual stylistic patterns. These findings indicate that safeguards targeting literal copying alone are insufficient to mitigate broader copyright risks. PSALM provides infrastructure for auditable, legally informed compliance evaluation, though the relationship between automated similarity scores and infringement determinations requires validation by legal experts. This work bridges qualitative legal standards and quantitative technical measurement, exposing fundamental tensions between generative AI and EU intellectual property law.