FairBED: A Bayesian Experimental Design Approach to Gathering Fairer Data

2026-06-22

AI Summary

This work proposes FairBED, a fairness-aware Bayesian experimental design framework that proactively optimizes sample selection during data acquisition to mitigate inherent biases in existing datasets. Rather than relying solely on post-hoc adjustments to biased data, FairBED formalizes dataset fairness as the non-inferability of sensitive attributes by minimizing information gain about them during sampling. The method establishes a theoretical connection between this information-theoretic notion of fairness and demographic parity. Integrating information-theoretic metrics with explicit fairness constraints, FairBED is evaluated across multiple benchmark datasets, demonstrating that models trained on data collected via this approach achieve significantly better fairness—measured against both random sampling and conventional Bayesian experimental design—while maintaining high predictive accuracy.

Abstract

Frameworks for ensuring fairness in machine learning typically focus on learning fair models from existing data. But this endeavor is often undermined by biases already present in that data. We therefore look to modify the data acquisition process itself to help gather fairer data that is inherently more suitable for training fair predictors. To this end, we introduce FairBED, which provides novel formulations for quantifying the fairness of datasets themselves based on the idea that fair datasets should be uninformative about sensitive attributes. We then use this to construct practical fairness-aware Bayesian experimental design (BED) objectives that maximize expected information gain about the target quantity of interest while minimizing expected information gain about sensitive attributes. We further derive a theoretical link between FairBED and demographic parity, and show empirically that models trained on data gathered using FairBED provide improved fairness-accuracy trade-offs compared to randomly acquired data and conventional BED.

Problem

• fairness
• data acquisition
• bias
• sensitive attributes
• fair machine learning

Innovation

• Bayesian experimental design
• fair data acquisition
• information gain minimization
• demographic parity
• fair machine learning