WIDER-FAIR: An Annotated Version of the WIDER-FACE Dataset for Fairness Evaluation

2026-06-30

AI Summary

Existing face detection datasets generally lack demographic annotations, making it difficult to evaluate model fairness across different population groups. This work addresses this gap by introducing systematic manual annotations—covering race (Asian, Black, Indian, White) and gender—into the large-scale WIDER FACE benchmark, resulting in a new dataset named WIDER-FAIR. Training and ablation studies based on YOLOv5 reveal significantly lower detection performance for Black individuals. Moreover, experiments show that removing this group from the training set exacerbates fairness disparities. The study provides a critical resource for fine-grained fairness analysis and highlights the substantial impact of underrepresented groups on model bias.

Abstract

The deployment of face detection models in real-world applications raises important fairness concerns, as these systems may showcase performance disparities across demographic groups. A key obstacle to studying and mitigating such biases is the lack of face detection datasets with sensitive feature annotations. To address this gap, we introduce WIDER-FAIR, a new dataset built on the widely used WIDER-FACE benchmark, manually annotated with the perceived ethnicity and sex of each face. The dataset contains 16,256 images annotated across four ethnic groups: Asian, Black, Indian, and White, and two sex categories. We assess the quality and coherence of the annotations using face embeddings, a K-Nearest Neighbors classifier, and a t-SNE visualization, all of which support the consistency of the labeling process. As a demonstration of the dataset's potential, we train a YOLOv5 model and perform ablation studies on each sensitive feature. Among other findings, our experiments show that detection performance is notably lower for faces of Black individuals, and that excluding this group from training increases fairness disparity more than excluding any other ethnic group. These observations illustrate the value of demographically annotated datasets for understanding and evaluating bias in face detection models.

Problem

• fairness
• face detection
• bias
• demographic annotation
• dataset

Innovation

• fairness evaluation
• demographic annotation
• face detection bias
• WIDER-FAIR dataset
• sensitive attributes