Search papers, labs, and topics across Lattice.
This paper critiques the effectiveness of traditional scoring functions in balancing utility and fairness in ranking systems, revealing that these methods are sub-optimal for achieving desired utility-fairness trade-offs. Through counter-examples, the authors demonstrate that both deterministic and randomized scoring functions fail to adequately address fairness across various query scopes. They propose a semi-greedy post-processing approach that significantly improves trade-offs, often nearing the performance of exhaustive methods while remaining computationally feasible.
Scoring functions can mislead in fairness-utility trade-offs, but a semi-greedy post-processing method can dramatically improve outcomes.
Scoring functions are used to represent the relevance of individual documents. In modern information retrieval or recommendation systems, they are often learned from data and play a pivotal role in ranking sets of documents or items in a way that maximizes utility to a query or user. With the recent interest in algorithmic fairness, the success of scoring has naturally led to methods that learn scores that simultaneously trade off fairness and utility. In this work, we show that in stark contrast with utility-centric objectives, scoring is sub-optimal in achieving all utility-fairness trade-offs. We establish this with a series of counter-examples with a generic fairness formulation. We show that the issue persists whether we have a deterministic scoring function or a randomized one, or whether we measure fairness at the scope of a single query or across multiple queries. On the positive side, we empirically demonstrate that semi-greedy post-processing has the potential to achieve much better trade-offs, often approaching the ideal of exhaustive post-processing in a tractable way.
