Search papers, labs, and topics across Lattice.
This paper conducts the first independent end-to-end fairness audit of a semi-automated hiring system used by Barcelona Activa, analyzing nearly 497,000 candidate-vacancy entries over five years. While aggregate outcomes suggest gender parity, significant disparities are revealed, particularly disadvantaging women in mid-salary shortlisting and non-binary candidates, alongside a complete absence of candidates aged 55 and over in the pipeline. The findings highlight the critical role of organizational context and vendor transparency in understanding fairness, emphasizing that model-level assessments alone are inadequate for evaluating deployed systems' fairness.
Despite appearing fair at first glance, the hiring system reveals deep-seated biases that disproportionately affect women and non-binary candidates, challenging the notion of algorithmic neutrality.
Algorithmic fairness evaluation commonly assesses AI systems as bounded technical components, abstracting away the organizational context in which they operate. We present, to our knowledge, the first independent end-to-end fairness audit of a semi-automated hiring system operated by Barcelona Activa, a public employment agency using the third-party TalentClue platform for candidate search and shortlisting. We analyze approximately 497,000 candidate-vacancy pipeline entries from September 2017 to September 2022, covering seven pipeline stages that span automated processing, human discretion, candidate data, and employer decisions. Aggregate outcomes across binary genders are statistically indistinguishable, yet this parity masks substantial disparities by salary level, age, and gender identity. Women face adverse impact in mid-salary shortlisting (DIR = 0.786, p<0.001), alongside salary disparities in 15 of 20 sectors and a compounded disadvantage for women aged 46-55 (DIR = 0.77). Non-binary candidates are shortlisted at less than one third the rate of men (DIR = 0.295), although this estimate rests on a small sample (N = 285). Candidates aged 55 and over are entirely absent from the pipeline despite comprising 15.6% of Barcelona's labor force. The gender gap in shortlisting narrows over time, from 6.5 percentage points in 2017 to 1.3 in 2022. The audit further reveals a vendor-deployer information asymmetry: Barcelona Activa lacks access to key information about TalentClue's matching logic and evaluation. Fairness outcomes can thus arise from interactions among automated processing, human discretion, data quality, vendor opacity, and pipeline structure. We build on prior calls for sociotechnical, end-to-end fairness evaluation, showing empirically why model-level assessment alone can be insufficient for understanding fairness in deployed systems.
