Search papers, labs, and topics across Lattice.
This study explores the vulnerabilities of machine-text detectors against evasion strategies, revealing that while current manipulation techniques can reduce detection efficacy, they do not eliminate the stylistic fingerprints inherent in machine-generated text. The authors introduce a novel paraphrasing method that successfully evades detection across various detectors, including those that leverage stylistic features. However, they also find that as the dataset size increases, the distinction between human and machine-generated text re-emerges, indicating that robust detection may require multi-document analysis rather than relying solely on single-document assessments.
Evasion strategies can bypass even advanced detectors by leveraging stylistic manipulation, but the effectiveness diminishes with larger datasets.
Despite considerable progress in the development of machine-text detectors, the ease with which machine-text can be manipulated to evade detection has led to suggestions that the problem is inherently intractable. In this work, we investigate the limits of such evasion strategies. We demonstrate that while current attacks, ranging from prompt engineering to detector-guided optimization can effectively degrade performance of standard detectors, they fail to erase the underlying stylistic "fingerprints" of machine text. We show that few-shot detectors that utilize the stylistic feature space are robust to these evasion attempts, reliably detecting samples even from models explicitly tuned to prevent detection. This raises the question: does style represent a universal defense against machine-detection attacks? We demonstrate that the answer is "no'' by introducing a novel paraphrasing approach that simultaneously optimizes for undetectability and adherence to specific human styles. We show that unlike prior methods, this attack effectively evades all considered detectors, including those that utilize writing style. However, we find that this evasion is not absolute: as the number of documents available for analysis grows, the human and machine distributions become distinguishable again. Overall, our findings suggest that reliable machine-text detection requires moving beyond single-document analysis to multi-document analysis.
