Search papers, labs, and topics across Lattice.
Luminol-AIDetect, a zero-shot MGT detection method, leverages the hypothesis that machine-generated text exhibits greater structural fragility under text shuffling compared to human writing due to the autoregressive nature of LLMs. The method calculates perplexity shifts after randomized text shuffling, extracting scalar features to differentiate between MGT and human text using density estimation and ensemble prediction. Evaluated across diverse domains and languages, Luminol-AIDetect achieves state-of-the-art performance, reducing FPR by up to 17x compared to existing methods.
Machine-generated text is surprisingly fragile: shuffling its words reveals a telltale perplexity signature that distinguishes it from human writing with state-of-the-art accuracy.
Machine-generated text (MGT) detection requires identifying structurally invariant signals across generation models, rather than relying on model-specific fingerprints. In this respect, we hypothesize that while large language models excel at local semantic consistency, their autoregressive nature results in a specific kind of structural fragility compared to human writing. We propose Luminol-AIDetect, a novel, zero-shot statistical approach that exposes this fragility through coherence disruption. By applying a simple randomized text-shuffling procedure, we demonstrate that the resulting shift in perplexity serves as a principled, model-agnostic discriminant, as MGT displays a characteristic dispersion in perplexity-under-shuffling that differs markedly from the more stable structural variability of human-written text. Luminol-AIDetect leverages this distinction to inform its decision process, where a handful of perplexity-based scalar features are extracted from an input text and its shuffled version, then detection is performed via density estimation and ensemble-based prediction. Evaluated across 8 content domains, 11 adversarial attack types, and 18 languages, Luminol-AIDetect demonstrates state-of-the-art performance, with gains up to 17x lower FPR while being cheaper than prior methods.
