Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a test-time adaptation (TTA) approach for AI text detection that addresses vulnerabilities caused by continual distribution shifts, such as adversarial humanization and new LLM releases. By leveraging the homogeneity of unlabeled samples at inference time through semi-supervised learning, the proposed method significantly outperforms state-of-the-art supervised detectors, achieving a detection rate of 90.5% on adversarial AI-generated text compared to just 24.1% for existing models. The findings highlight the robustness of TTA in real-world applications, where labeled data is often scarce or unavailable.
Test-time adaptation can boost AI text detection accuracy from 24.1% to 90.5% under adversarial conditions, revealing a critical vulnerability in current models.
Deployed approaches for AI text detection often rely on training-time access to labeled datasets of both human-written and AI-generated text. This approach is vulnerable to three types of distribution shifts that occur continually post-deployment, and for which labeled data is often unavailable: adversarial humanization, new LLMs being released, and temporal drift in human writing. Simultaneously, existing approaches do not leverage a key signal of LLM usage: inference-time homogeneity. We propose a test-time adaptation (TTA) approach, using semi-supervised learning, that adapts to distribution shifts by leveraging homogeneity among unlabeled samples observed at inference time. Empirically, we find that state-of-the-art supervised detectors systematically fail when they encounter distribution shifts in AI-generated and human writing, both adversarial and natural, while test-time adaptation with semi-supervised learning is largely robust; e.g., the commercial model Pangram detects just 24.1% of our adversarial AI-generated text, compared to 90.5% for our test-time approach. We establish that test-time adaptation is a promising framework for AI text detection in the wild. We publicly release our code (which includes code for model training, evaluation, and plots) at https://github.com/kkr36/llm_detection.