Search papers, labs, and topics across Lattice.
This paper critiques the reliance on Equal Error Rate (EER) for evaluating voice anonymisation, highlighting its inadequacy in addressing real-world 1-to-N linkage threats. The authors introduce a modular, information-theoretic framework that includes a new metric, Local Information Disclosure (LID), which quantifies privacy loss in bits by assessing the attacker's confidence in identifying a speaker from a set of enrolled identities. Their findings reveal that systems with high EER can still exhibit significant localized vulnerabilities, underscoring the necessity of adopting localized metrics for a more accurate assessment of privacy risks.
Systems with near-perfect EER can still expose significant privacy vulnerabilities, with worst-case disclosures effectively doubling an attacker's success rate.
Voice anonymisation aims to protect speaker identity. Currently, its empirical privacy evaluation heavily relies on the Equal Error Rate (EER). Originally designed for biometric verification, EER aggregates scores globally, implicitly assuming an attacker is only trying to verify if two specific voice samples match (a 1-to-1 comparison). This introduces a threat model mismatch with real-world database linkage attacks, where an attacker searches across a fixed set of N enrolled identities (a 1-to-N closed-set search), allowing global averages to obscure localised privacy failures. While recent 1-to-N metrics address this aggregation issue, they abstract away the magnitude of the biometric evidence. In this paper, we propose a modular, information-theoretic evaluation framework explicitly designed for the 1-to-N linkage threat model. Within this framework, our core metric, Local Information Disclosure (LID), quantifies the exact privacy loss of a single trial utterance in bits by calibrating its raw similarity scores into the attacker's posterior confidence for each enrolled identity. Evaluating top-performing systems from the VoicePrivacy 2024 Challenge reveals that systems exhibiting near-perfect EERs (48 %) can still suffer from localised vulnerabilities with worst-case disclosures reaching 1 bit per trial utterance (effectively doubling the attacker's success rate over a random guess). We demonstrate that adopting localised privacy metrics is essential for capturing worst-case risks and aligning with strict privacy regulations.
