Bias Smells in AI Software Development: Recognizing Potential Sources of Fairness Debt

2026-08-10

AI Summary

This study addresses the challenge of early identification of fairness risks in AI software development by introducing the concept of “bias smells,” extending the software smell paradigm to the domain of algorithmic fairness. Through an exploratory case study, semi-structured interviews, open-ended surveys, and document analysis—analyzed via constant comparative qualitative coding—the authors identify six actionable bias smells: contextual oversimplification, dataset imbalance, inadequate metrics, ad hoc testing, neglect of individual diversity, and team homogeneity. This framework bridges technical practices with societal impacts, offering an empirically grounded foundation for ensuring fairness in AI systems and enabling early-stage interventions in the development lifecycle.

Abstract

Context: Fairness debt arises from AI development shortcomings that may lead to societal harms. While technical and social debt concern software design decisions and team dynamics, respectively, fairness debt captures the long-term consequences of development decisions that may reinforce bias and inequities in AI-based software systems. Despite growing attention to AI fairness, limited evidence exists on how practitioners recognize potential sources of fairness debt during development. Aim: This study investigates the indicators practitioners recognize as signaling potential sources of fairness debt in AI-based software projects. Method: We conducted an exploratory case study of four AI projects within one organization. Data were collected from 25 professionals through semi-structured interviews and open-ended questionnaires, complemented by observation of internal communication channels and project documentation, and analyzed using iterative qualitative coding, memoing, and constant comparison. Results: We identified six recurring indicators, termed bias smells: Context Oversimplification, Dataset Imbalance, Metrics Inadequacies, Ad hoc Testing, Individual Diversity Unawareness, and Homogeneous Team Composition. These smells span technical and human aspects of software development and signal conditions that may introduce or reinforce bias and contribute to fairness debt. Conclusion: Bias smells extend the software smell paradigm to fairness and provide a foundation for incorporating fairness into software quality assurance through observable indicators.

Problem

• fairness debt
• bias smells
• AI fairness
• software development
• societal harms

Innovation

• bias smells
• fairness debt
• AI fairness
• software quality assurance
• qualitative case study