Scoring Is Not Enough: Addressing Gaps in Utility-fairness Trade-offs for Ranking

2026-06-24

AI Summary

Traditional scoring functions exhibit inherent theoretical limitations in balancing utility and fairness for ranking tasks. This work rigorously proves, for the first time, that such functions cannot span the entire Pareto frontier of utility–fairness trade-offs. The universality of this limitation is demonstrated through counterexamples constructed across multiple settings, including deterministic versus stochastic scenarios and single-query versus multi-query contexts. To address this gap, the paper introduces a semi-greedy post-processing algorithm that efficiently approximates the ideal solution within a general formal fairness framework. Experimental results show that the proposed method substantially outperforms existing scoring mechanisms and achieves performance close to that of exhaustive post-processing within computationally feasible bounds.

Abstract

Scoring functions are used to represent the relevance of individual documents. In modern information retrieval or recommendation systems, they are often learned from data and play a pivotal role in ranking sets of documents or items in a way that maximizes utility to a query or user. With the recent interest in algorithmic fairness, the success of scoring has naturally led to methods that learn scores that simultaneously trade off fairness and utility. In this work, we show that in stark contrast with utility-centric objectives, scoring is sub-optimal in achieving all utility-fairness trade-offs. We establish this with a series of counter-examples with a generic fairness formulation. We show that the issue persists whether we have a deterministic scoring function or a randomized one, or whether we measure fairness at the scope of a single query or across multiple queries. On the positive side, we empirically demonstrate that semi-greedy post-processing has the potential to achieve much better trade-offs, often approaching the ideal of exhaustive post-processing in a tractable way.

Problem

• scoring
• utility-fairness trade-offs
• ranking
• algorithmic fairness
• information retrieval

Innovation

• scoring functions
• utility-fairness trade-offs
• algorithmic fairness
• semi-greedy post-processing
• ranking systems