Search papers, labs, and topics across Lattice.
This paper investigates potential biases in AI content watermarking across different modalities (text, image, audio) by examining how watermark signal strength varies across languages, cultural content, and demographic groups. The authors find that existing watermarking benchmarks largely fail to report performance across these diverse dimensions, leading to potential disparities in detectability and robustness. They propose three evaluation dimensions for pluralistic watermark benchmarking: cross-lingual detection parity, culturally diverse content coverage, and demographic disaggregation of detection metrics, arguing for bias auditing requirements similar to those applied to generative models.
Watermarking, intended to govern AI content, is held to a lower fairness standard than the AI models it's meant to regulate, potentially leading to biased content authentication.
Watermarking is becoming the default mechanism for AI content authentication, with governance policies and frameworks referencing it as infrastructure for content provenance. Yet across text, image, and audio modalities, watermark signal strength, detectability, and robustness depend on statistical properties of the content itself, properties that vary systematically across languages, cultural visual traditions, and demographic groups. We examine how this content dependence creates modality-specific pathways to bias. Reviewing the major watermarking benchmarks across modalities, we find that, with one exception, none report performance across languages, cultural content types, or population groups. To address this, we propose three concrete evaluation dimensions for pluralistic watermark benchmarking: cross-lingual detection parity, culturally diverse content coverage, and demographic disaggregation of detection metrics. We connect these to the governance frameworks currently mandating watermarking deployment and show that watermarking is held to a lower fairness standard than the generative systems it is meant to govern. Our position is that evaluation must precede deployment, and that the same bias auditing requirements applied to AI models should extend to the verification layer.
