Search papers, labs, and topics across Lattice.
This study explores how AI practitioners identify potential sources of fairness debt in software development, which can lead to societal harms due to biased AI systems. Through an exploratory case study involving 25 professionals across four AI projects, the researchers identified six key indicators, termed "bias smells," that signal conditions likely to introduce or reinforce bias. These findings extend the existing software smell paradigm to encompass fairness, providing actionable insights for improving software quality assurance in AI development.
Six key "bias smells" can help practitioners recognize and mitigate fairness debt in AI software development before it leads to societal harm.
Context: Fairness debt arises from AI development shortcomings that may lead to societal harms. While technical and social debt concern software design decisions and team dynamics, respectively, fairness debt captures the long-term consequences of development decisions that may reinforce bias and inequities in AI-based software systems. Despite growing attention to AI fairness, limited evidence exists on how practitioners recognize potential sources of fairness debt during development. Aim: This study investigates the indicators practitioners recognize as signaling potential sources of fairness debt in AI-based software projects. Method: We conducted an exploratory case study of four AI projects within one organization. Data were collected from 25 professionals through semi-structured interviews and open-ended questionnaires, complemented by observation of internal communication channels and project documentation, and analyzed using iterative qualitative coding, memoing, and constant comparison. Results: We identified six recurring indicators, termed bias smells: Context Oversimplification, Dataset Imbalance, Metrics Inadequacies, Ad hoc Testing, Individual Diversity Unawareness, and Homogeneous Team Composition. These smells span technical and human aspects of software development and signal conditions that may introduce or reinforce bias and contribute to fairness debt. Conclusion: Bias smells extend the software smell paradigm to fairness and provide a foundation for incorporating fairness into software quality assurance through observable indicators.
