Search papers, labs, and topics across Lattice.
The paper introduces GhostWriteBench, a dataset of long-form texts generated by LLMs, designed to evaluate authorship attribution across diverse out-of-distribution scenarios. They propose TRACE, a novel fingerprinting method that captures token-level transition patterns using a lightweight language model to identify the LLM author. Experiments on GhostWriteBench show TRACE achieves state-of-the-art performance, robustness in OOD settings, and effectiveness with limited training data.
You can now unmask LLM ghostwriters with a lightweight fingerprinting method that works even when they try to hide in new domains or use unseen models.
In this paper, we introduce GhostWriteBench, a dataset for LLM authorship attribution. It comprises long-form texts (50K+ words per book) generated by frontier LLMs, and is designed to test generalisation across multiple out-of-distribution (OOD) dimensions, including domain and unseen LLM author. We also propose TRACE -- a novel fingerprinting method that is interpretable and lightweight -- that works for both open- and closed-source models. TRACE creates the fingerprint by capturing token-level transition patterns (e.g., word rank) estimated by another lightweight language model. Experiments on GhostWriteBench demonstrate that TRACE achieves state-of-the-art performance, remains robust in OOD settings, and works well in limited training data scenarios.
