Search papers, labs, and topics across Lattice.
This paper develops a geometric framework to analyze the robustness of neighborhood-based fairness audits, which assess individual fairness by comparing predictions among similar individuals in feature space. The authors establish sufficient conditions for neighborhood invariance and introduce a novel measure called audit volatility to quantify the sensitivity of fairness audits to perturbations. Experimental results on benchmark datasets validate the theoretical findings, demonstrating that the proposed framework can explain the stability of these audits despite local neighborhood changes.
Neighborhood-based fairness audits can be surprisingly unstable, with small perturbations leading to significant shifts in fairness assessments.
Neighborhood-based fairness audits evaluate individual fairness by comparing predictions among similar individuals in feature space. Despite their widespread use, little is known about the robustness of the auditing procedure itself. Because these audits rely on nearest neighbor relationships, small perturbations in feature space can alter local neighborhoods and produce different fairness assessments even when model predictions remain unchanged. We develop a geometric framework for analyzing the robustness of neighborhood-based fairness audits under bounded perturbations. Our analysis establishes sufficient conditions for neighborhood invariance, quantifies how neighborhood replacement propagates to audit instability, and introduces audit volatility, a measure of the expected sensitivity of fairness audits under repeated perturbations. Experiments on benchmark datasets support the theoretical analysis and show that the proposed framework explains the observed stability of neighborhood-based fairness audits.
