Search papers, labs, and topics across Lattice.
This paper investigates the vulnerability of watermarking techniques used in AI-generated text, revealing that accessing multiple models leads to the trivial failure of these watermarks due to independent perturbations in output distributions. The authors theoretically prove that averaging the outputs of multiple models can effectively recover the original unwatermarked distribution, and empirically demonstrate that averaging just 3-5 models significantly reduces detection capabilities while improving output quality and efficiency. Their method, WASH (Watermark Attenuation via Statistical Hybridisation), addresses practical challenges in ensemble generation, achieving remarkable results across various watermarking schemes and LLMs.
Averaging outputs from multiple models can completely nullify watermark detection, raising critical questions about the future of AI text attribution.
Watermarking embeds statistical signatures in AI-generated text for detection and attribution. We reveal a fundamental vulnerability: when users access multiple models (today's reality), watermarks trivially fail. Watermarks perturb output distributions away from the original, and in competitive markets, these perturbations are typically independent across providers. We theoretically prove that averaging output probability distributions recovers the unwatermarked distribution with up to a second-order error term. Empirically, simply averaging 3-5 models cancels out these perturbations. We introduce WASH (Watermark Attenuation via Statistical Hybridisation), which solves practical challenges in ensemble generation: vocabulary misalignment and tokenisation differences across heterogeneous models. Experiments across six watermarking schemes and three LLMs show that averaging across 3 models suppresses detection z-scores from 5-300 to below 2 (below the detection threshold of 4) and reduces TPR at 5% FPR to below 50%, while improving quality by 27.5% and running 6 times faster than the best baseline on the long sequence generation. Our results suggest that robust AI-text detection via watermarking requires either accepting this fundamental vulnerability or unprecedented coordination among model providers.
