FairSelect: A Systematic Evaluation of Multi-Level and Intersectional Algorithmic Fairness

2026-07-09

AI Summary

This work addresses the limitations of existing fairness methods, which often focus on a single demographic attribute and lack systematic evaluation across intersecting subgroups and multiple stages of the modeling pipeline. To bridge this gap, we propose FairSelect, a novel toolkit that establishes the first multi-level evaluation framework enabling arbitrary combinations of pre-, in-, and post-processing fairness interventions. We conduct comprehensive analyses of fairness–utility trade-offs across diverse model architectures and intersectional subgroups using both synthetic clinical data and a real-world atrial fibrillation stroke risk prediction task. Our experiments demonstrate that combined intervention strategies generally enhance fairness with controllable utility loss; notably, certain combinations simultaneously improve both fairness and predictive performance, while others yield adverse effects, revealing non-additive and context-dependent interactions among fairness interventions in intersectional settings.

Abstract

Algorithmic fairness methods are increasingly used to identify and mitigate bias in machine learning models, yet most approaches are evaluated in isolation and along single demographic axes. This limits practical guidance for selecting fairness strategies, where disparities may arise across intersectional subgroups and across multiple stages of the modeling lifecycle. This work presents FairSelect, a toolkit for systematically evaluating fairness mitigation strategies applied individually and in combination across preprocessing, inprocessing, and postprocessing stages. FairSelect supports multiple model architectures, intersectional subgroup evaluation, and comparison of fairness utility tradeoffs across baseline, single method, and multi level configurations. The framework was validated using synthetic clinical datasets designed to represent specific bias mechanisms and a real-world replication of two-year stroke risk prediction among patients with atrial fibrillation. Synthetic experiments showed that targeted fairness methods generally reduced intended subgroup disparities, while combined strategies produced larger average fairness improvements with modest utility tradeoffs. In the clinical prediction task, mitigation effects were highly variable, with some combinations improving both fairness and predictive performance while others were ineffective or counterproductive. These findings demonstrate that fairness interventions interact in nonadditive and context dependent ways. FairSelect provides a practical framework for systematically identifying fairness strategies that improve subgroup equity while preserving model performance in clinical machine learning.

Problem

• algorithmic fairness
• intersectional fairness
• fairness mitigation
• clinical machine learning
• multi-level evaluation

Innovation

• algorithmic fairness
• intersectionality
• fairness mitigation
• multi-level evaluation
• fairness-utility tradeoff