dataset bias auditing

This work addresses algorithmic bias in AI models arising from implicit sensitive attributes—such as gender, age, and race—in image datasets. To this end, we propose an end-to-end fairness auditing framework. Methodologically: (1) we design a lightweight CNN dedicated to extracting sensitive attribute features; (2) we introduce a novel robust statistical test that integrates prediction uncertainty to quantify distributional disparities across demographic subgroups; and (3) we adopt a weakly supervised paradigm, requiring only a small number of human annotations to fully automate the auditing pipeline. Experiments on two facial benchmark datasets demonstrate that our approach significantly improves both the accuracy and robustness of bias detection. The framework provides a reproducible, scalable, and technically grounded solution for regulatory compliance auditing of AI systems, enabling rigorous, data-driven fairness assessment without extensive labeling effort.

This paper addresses selection and label bias in fairness evaluation for automated decision-making (e.g., hiring), which leads mainstream interventions—such as resampling—to produce a “fairness illusion.” To mitigate this, we propose an audit-inspired fictitious applicant experimental framework integrating randomized controlled trials, base-rate parity, and individual treatment effect (ITE) estimation. Our method enables more realistic fairness assessment and model training. Key contributions include: (i) the first incorporation of ITE estimation into algorithmic fairness intervention, thereby reducing overreliance on aggregate group-level metrics; and (ii) empirical evidence demonstrating that conventional methods conceal approximately 10% residual discrimination, whereas our approach significantly reduces actual discriminatory outcomes. The framework enhances internal validity of fairness evaluation and improves intervention efficacy.

This study addresses the critical gap in clinical machine learning fairness evaluation by systematically applying an intersectional fairness auditing framework to real-world clinical prediction tasks. Leveraging the All of Us dataset, the authors integrate the FairLogue toolkit, observational fairness metrics, and counterfactual causal analysis to assess model performance across intersecting subgroups defined by race and gender. Their findings reveal substantial performance disparities that remain undetected under conventional single-axis fairness assessments. However, counterfactual experiments demonstrate that most of these disparities persist even after randomizing group identity, indicating that they primarily stem from differences in covariate distributions rather than direct discrimination. These results underscore the necessity and value of intersectional auditing for accurately diagnosing and addressing health inequities in clinical AI systems.

This study systematically audits implicit racial and gender biases—particularly against Black women—in large language models (LLMs) such as GPT-4 within named-individual scenarios (e.g., car negotiation, election forecasting). We propose the first adversarial fairness auditing framework grounded in 42 structured prompt templates, enabling cross-model comparative analysis and quantitative attribution. Our empirical analysis reveals, for the first time, highly consistent race–gender interaction bias at the name level across models and templates. Crucially, we demonstrate that introducing numeric anchoring significantly mitigates bias, reducing adverse recommendation rates by up to 37%—challenging prevailing qualitative prompt-tuning practices. The methodology establishes a reproducible, scalable technical pathway for pre-deployment fairness evaluation of LLMs.

To address the challenge of sustaining fairness auditing for deployed models under data policy drift, subgroup heterogeneity, and dynamic distributional shifts, this paper proposes the first dynamic fairness auditing framework tailored for streaming data. Methodologically, it innovatively adapts “testing by betting”—a game-theoretic statistical paradigm—into fairness monitoring, integrating anytime-valid inference with nonparametric, sample-size-agnostic sequential testing that adapts to time-varying sampling strategies. Unlike conventional batch-based approaches, our framework enables real-time, continuous fairness assessment without requiring prior knowledge of data distribution or fixed sample sizes. Experiments on three benchmark datasets demonstrate that the method achieves high detection sensitivity, low false positive rate, and strong robustness against concept drift and subgroup imbalance. These properties collectively enhance the practicality, timeliness, and deployability of fairness monitoring in real-world operational environments.

Current model evaluations often rely on aggregate metrics that obscure performance disparities and unfairness across continuous or fine-grained subpopulations. This work proposes FairTree, an algorithm that introduces bias-variance decomposition into fairness auditing for the first time, drawing inspiration from measurement invariance in psychometrics to handle continuous, categorical, and ordinal attributes without requiring discretization. By integrating permutation tests with fluctuation tests, FairTree flexibly models subpopulation performance variation and enables rigorous statistical inference. Empirical results demonstrate that FairTree effectively controls false positive rates, with the fluctuation test exhibiting superior statistical power, and its practical utility is validated on the UCI Adult Census dataset.

This study conducts a third-party fairness audit of LinkedIn’s talent search ranking system, focusing on gender- and race-based ranking bias. Addressing the limitation of prior work—which predominantly assesses static exposure while neglecting temporal dynamics—we propose a two-dimensional fairness framework: “static exposure” and “dynamic stability.” We quantify exposure inequality using group proportion deviation and MinSkew, and evaluate stability via ranking volatility and positional persistence. Leveraging large-scale real-world query logs and proxy-based sensitive attribute inference, we perform an external, black-box audit. Empirical results reveal significant underrepresentation of women and racial minorities at top-ranking positions, coupled with higher ranking instability and shorter positional retention—indicating systemic disadvantages in ranking durability. To our knowledge, this is the first fairness evaluation of recruitment platforms to explicitly incorporate temporal dimensions, offering a novel paradigm and empirical foundation for sustainable, algorithmic fairness governance in hiring systems.

This work addresses the challenge of auditing external fairness in real-world AI systems, where auditors typically have only limited query access to model outputs. Existing approaches rely on static datasets and are ill-suited for sequential auditing scenarios. To bridge this gap, the paper introduces the first sequential fairness auditing framework tailored to practical deployment constraints, formulating the audit as a tolerance-aware sequential hypothesis test. Built upon generalized likelihood ratio methods, the framework accommodates diverse model outputs—including decisions, scores, and logits—and dynamically accumulates evidence, enabling early termination once a conclusive determination is reached. Empirical results demonstrate that both the type of fairness metric and the richness of model output significantly influence auditing efficiency: richer outputs generally reduce query complexity substantially, though their advantage diminishes when the system’s fairness level is near the decision threshold.

This study addresses the vulnerability of fairness auditing in resource-constrained settings, where computationally unbounded firms may manipulate their models post-audit to circumvent fairness constraints, thereby undermining fairness guarantees in high-stakes applications. The problem is formalized as a minimax optimization between a resource-limited auditor and a strategic firm, yielding the first quantification of the fundamental lower bound on post-audit manipulation under a finite auditing budget. The theoretical analysis integrates group imbalance and tolerance parameters, deriving a worst-case lower bound on demographic parity violation that jointly depends on the audit budget, degree of group imbalance, and allowable tolerance. Empirical evaluations on both linear models and neural networks confirm that while increased auditing resources can mitigate manipulation, they cannot fully eliminate the strategic firm’s ability to exploit residual gaps.

This work addresses the challenge of systematic performance disparities exhibited by machine learning models across sensitive subgroups in high-stakes settings, where existing fairness auditing methods often rely on strong distributional assumptions or incur prohibitive computational costs. The paper introduces, for the first time, empirical likelihood into fairness auditing, proposing a nonparametric, distribution-free statistical inference framework. By constructing a discrepancy statistic that asymptotically follows a mixture of chi-squared distributions, the method enables efficient and accurate fairness certification and subgroup bias detection without resampling. Demonstrating both computational efficiency and statistical power, the approach successfully identifies intersectional biases on the COMPAS dataset, achieves coverage matching nominal confidence levels, and accelerates computation by several orders of magnitude compared to bootstrap methods, making it suitable for large-scale deployment.