RISE: Interactive Visual Diagnosis of Fairness in Machine Learning Models

2026-02-04

AI Summary

This work addresses the challenge of identifying bias sources under domain shift, where conventional scalar fairness metrics often fall short. The authors propose RISE, a novel diagnostic framework that, for the first time, formally links ranking residual curves with rigorous fairness notions to enable fine-grained and interactive fairness analysis. By leveraging residual ranking, visualization, and post-hoc examination, RISE facilitates localized fairness evaluation, cross-environment subgroup comparisons, and the discovery of hidden biases. The framework effectively uncovers accuracy–fairness trade-offs obscured by aggregate statistics, thereby empowering practitioners to make more informed and equitable model selection decisions.

Technology Category

• Computer Vision: Bias, Fairness & Privacy
• Machine Learning: Ethics, Bias, and Fairness
• Philosophy and Ethics of AI: Bias, Fairness & Equity

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

Evaluating fairness under domain shift is challenging because scalar metrics often obscure exactly where and how disparities arise. We introduce \textit{RISE} (Residual Inspection through Sorted Evaluation), an interactive visualization tool that converts sorted residuals into interpretable patterns. By connecting residual curve structures to formal fairness notions, RISE enables localized disparity diagnosis, subgroup comparison across environments, and the detection of hidden fairness issues. Through post-hoc analysis, RISE exposes accuracy-fairness trade-offs that aggregate statistics miss, supporting more informed model selection.

Problem

• fairness
• domain shift
• machine learning
• disparity diagnosis
• model evaluation

Innovation

• interactive visualization
• fairness diagnosis
• residual analysis
• domain shift
• model interpretability