Benchmarking Stochastic Approximation Algorithms for Fairness-Constrained Training of Deep Neural Networks

2025-07-05

AI Summary

Current fairness-constrained training of deep neural networks (DNNs) lacks a unified, reproducible evaluation benchmark. This paper introduces the first large-scale, real-world fairness-constrained learning benchmark, built upon Folktables—a U.S. Census-derived dataset—and systematically evaluates stochastic approximation-based constrained optimization algorithms for non-convex DNN training, focusing on their efficacy and accuracy–fairness trade-offs. Methodologically, we integrate state-of-the-art techniques—including stochastic gradient projection, dual updates, and constraint relaxation—to enable standardized comparison across three algorithmic families. Our contributions are threefold: (1) identifying theoretical challenges in large-scale non-convex constrained optimization; (2) releasing the first open-source, modular, and extensible fairness training benchmark; and (3) empirically characterizing algorithmic differences along the accuracy–fairness Pareto frontier, establishing a reproducible foundation for future research.

Technology Category

• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Search and Optimization: Non-convex Optimization
• Machine Learning: Ethics, Bias, and Fairness

Application Category

• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

The ability to train Deep Neural Networks (DNNs) with constraints is instrumental in improving the fairness of modern machine-learning models. Many algorithms have been analysed in recent years, and yet there is no standard, widely accepted method for the constrained training of DNNs. In this paper, we provide a challenging benchmark of real-world large-scale fairness-constrained learning tasks, built on top of the US Census (Folktables). We point out the theoretical challenges of such tasks and review the main approaches in stochastic approximation algorithms. Finally, we demonstrate the use of the benchmark by implementing and comparing three recently proposed, but as-of-yet unimplemented, algorithms both in terms of optimization performance, and fairness improvement. We release the code of the benchmark as a Python package at https://github.com/humancompatible/train.

Problem

• Benchmarking fairness-constrained DNN training algorithms
• Comparing optimization and fairness performance of three algorithms
• Providing a large-scale real-world fairness benchmark dataset

Innovation

• Benchmarking fairness-constrained DNN training
• Comparing stochastic approximation algorithms
• Implementing three novel fairness algorithms