Evaluating Fairness and Mitigating Bias in Machine Learning: A Novel Technique using Tensor Data and Bayesian Regression

2025-06-13

AI Summary

This work addresses the challenge of fairness evaluation in image classification arising from skin tone as a continuous, high-dimensional tensor-valued sensitive attribute. To overcome reliance on discrete, manually annotated skin-tone categories, we propose a novel fairness modeling paradigm that requires no skin-tone labels. Our method represents skin tone as a probability distribution—rather than a scalar or class label—and quantifies both inter- and intra-group fine-grained fairness via the Wasserstein distance. We integrate unsupervised skin-tone representation learning, tensor embedding, and Bayesian regression with a polynomial kernel to mitigate bias. Crucially, this is the first approach to model skin tone distributionally, enabling fairness measurement and optimization over a continuous spectrum. Experiments on multiple benchmark datasets demonstrate substantial improvements in skin-tone spectral prediction parity: average inter-group accuracy disparity decreases by 37%, while eliminating dependence on human-labeled skin-tone annotations entirely.

Technology Category

• Computer Vision: Bias, Fairness & Privacy
• Machine Learning: Ethics, Bias, and Fairness
• Philosophy and Ethics of AI: Bias, Fairness & Equity

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

Fairness is a critical component of Trustworthy AI. In this paper, we focus on Machine Learning (ML) and the performance of model predictions when dealing with skin color. Unlike other sensitive attributes, the nature of skin color differs significantly. In computer vision, skin color is represented as tensor data rather than categorical values or single numerical points. However, much of the research on fairness across sensitive groups has focused on categorical features such as gender and race. This paper introduces a new technique for evaluating fairness in ML for image classification tasks, specifically without the use of annotation. To address the limitations of prior work, we handle tensor data, like skin color, without classifying it rigidly. Instead, we convert it into probability distributions and apply statistical distance measures. This novel approach allows us to capture fine-grained nuances in fairness both within and across what would traditionally be considered distinct groups. Additionally, we propose an innovative training method to mitigate the latent biases present in conventional skin tone categorization. This method leverages color distance estimates calculated through Bayesian regression with polynomial functions, ensuring a more nuanced and equitable treatment of skin color in ML models.

Problem

• Evaluating fairness in ML for image classification without annotation
• Handling tensor data like skin color via probability distributions
• Mitigating biases in skin tone categorization using Bayesian regression

Innovation

• Uses tensor data for skin color representation
• Applies Bayesian regression with polynomial functions
• Converts skin color into probability distributions