Search papers, labs, and topics across Lattice.
This study investigates the latent representations of machine-generated text (MGT) and human-written text (HWT), revealing that they are linearly separable in low-dimensional space. By employing simple linear probes, the authors demonstrate significant improvements in out-of-domain detection, achieving an 11 AUC increase while requiring fewer than 100 samples for near-peak performance. The findings suggest that linear probes can effectively capture a continuous spectrum of "machineness," providing a robust and efficient method for distinguishing between MGT and HWT across diverse contexts.
Simple linear probes can outperform complex detectors in distinguishing machine-generated text from human-written text, even with minimal training data.
Distinguishing machine-generated text (MGT) from human-written text (HWT) becomes increasingly important due to potential misuse. However, most supervised detectors often degrade out-of-domain (OOD) and require large, diverse training sets. In this work, we analyze the linearity and quality of MGT representations and show that simple linear probes outperform a wide range of detectors while being substantially more sample-efficient. We first show that MGT and HWT latent representations are linearly separable in low-dimensional space, and provide a plausible explanation for this separability through systematic differences in their representation quality. Motivated by these insights, we train two variants of simple linear probes and evaluate them across 4 benchmarks against 16 baselines. Probes consistently improve OOD detection (+11 AUC), requiring solely ${<}100$ samples to reach near-peak performance. We show that this transferability arises because probes recover a shared latent MGT direction that generalizes across diverse settings. Finally, we demonstrate that probing vectors capture a continuous spectrum of ``machineness'', highlighting their potential for fine-grained estimation of AI-edited text. Overall, our work provides insights into latent-space differences between MGT and HWT and demonstrates the potential of linear probes as as robust and sample-efficient MGT detectors. We release our code on~\href{https://github.com/gerritq/mgt_probes}{github}.
