On the Reliability of Cue Conflict and Beyond

2026-03-11

AI Summary

Existing methods for evaluating shape–texture bias suffer from entangled cues, ratio-based metrics that obscure absolute sensitivity, and category selection biases, leading to unstable and poorly interpretable results. This work proposes the REFINED-BIAS framework, which establishes a unified and reproducible evaluation paradigm by explicitly disentangling shape and texture cues, synthesizing balanced image pairs recognizable by both humans and models, and quantifying cue sensitivity through ranking-based metrics over the full label space. REFINED-BIAS systematically addresses these three longstanding limitations for the first time, enabling more accurate and interpretable bias diagnosis across diverse model architectures and training strategies, and thereby resolving empirical contradictions observed in prior studies.

Technology Category

• Computer Vision: Bias, Fairness & Privacy
• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

Understanding how neural networks rely on visual cues offers a human-interpretable view of their internal decision processes. The cue-conflict benchmark has been influential in probing shape-texture preference and in motivating the insight that stronger, human-like shape bias is often associated with improved in-domain performance. However, we find that the current stylization-based instantiation can yield unstable and ambiguous bias estimates. Specifically, stylization may not reliably instantiate perceptually valid and separable cues nor control their relative informativeness, ratio-based bias can obscure absolute cue sensitivity, and restricting evaluation to preselected classes can distort model predictions by ignoring the full decision space. Together, these factors can confound preference with cue validity, cue balance, and recognizability artifacts. We introduce REFINED-BIAS, an integrated dataset and evaluation framework for reliable and interpretable shape-texture bias diagnosis. REFINED-BIAS constructs balanced, human- and model- recognizable cue pairs using explicit definitions of shape and texture, and measures cue-specific sensitivity over the full label space via a ranking-based metric, enabling fairer cross-model comparisons. Across diverse training regimes and architectures, REFINED-BIAS enables fairer cross-model comparison, more faithful diagnosis of shape and texture biases, and clearer empirical conclusions, resolving inconsistencies that prior cue-conflict evaluations could not reliably disambiguate.

Problem

• cue conflict
• shape-texture bias
• neural network interpretability
• visual cues
• bias estimation

Innovation

• cue-conflict
• shape-texture bias
• REFINED-BIAS
• interpretable evaluation
• visual cues