"Trust Junk" Leads to Unjustified Support for Highly Discriminatory Predictive Models

2026-07-14

AI Summary

This study investigates how seemingly plausible yet redundant or irrelevant visual explanations in explainable artificial intelligence (XAI) can lead users to overtrust models that exhibit clear discriminatory behavior. Through a crowdsourced behavioral experiment, the authors systematically evaluate shifts in user trust when exposed to accurate but task-irrelevant explanatory information alongside biased model decisions. The findings reveal a “trust garbage” effect: even when a model’s outputs display overt bias, the presence of formally coherent but substantively irrelevant explanations significantly increases users’ trust and positive assessments of the model. This phenomenon underscores a critical rhetorical risk embedded in XAI design—namely, that poorly constructed explanations may inadvertently mask algorithmic unfairness. The work thus calls for greater caution in crafting explanations to ensure they do not obscure, but rather illuminate, a model’s unjust behaviors.

Abstract

The persuasive power of data visualizations can go awry: for instance, in an explainable AI (XAI) context, visualizations can produce over-trust of predictive models. In this paper, we use a crowdsourced study to show that providing accurate (but superfluous or irrelevant) data in a model explanation can, in fact, result in unjustified trust and other positive beliefs about a model, even when the model is patently discriminatory and unfair. Our results suggest that XAI designers and developers need to consider the implicit or explicit rhetorics of their work, and beware of the potential of visualizations to imbue models with unearned trust.

Problem

• explainable AI
• data visualization
• algorithmic fairness
• trust
• discriminatory models

Innovation

• Trust Junk
• Explainable AI
• Model Discrimination
• Data Visualization
• Unjustified Trust