Err on the Side of Texture: Texture Bias on Real Data

2024-12-13

arXiv.org

AI Summary

Deep image classifiers often exhibit poor robustness due to over-reliance on texture cues rather than shape, leading to high-confidence misclassifications on natural adversarial examples. Method: We propose the Texture-Association Value (TAV)—the first interpretable, quantitative metric for measuring texture–class association strength directly on real-world images. TAV integrates feature attribution with statistical significance testing, coupled with large-scale real-distribution modeling and error-pattern mining to systematically identify texture mismatch as the primary cause of natural adversarial examples. Results: Experiments reveal that over 90% of natural adversarial examples suffer from texture–label mismatches, directly linking texture bias to confident mispredictions and robustness collapse. This work establishes the first empirically grounded, measurable, attributable, and verifiable framework for diagnosing texture bias—providing a novel pathway toward improving model generalization and trustworthiness.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Bias significantly undermines both the accuracy and trustworthiness of machine learning models. To date, one of the strongest biases observed in image classification models is texture bias-where models overly rely on texture information rather than shape information. Yet, existing approaches for measuring and mitigating texture bias have not been able to capture how textures impact model robustness in real-world settings. In this work, we introduce the Texture Association Value (TAV), a novel metric that quantifies how strongly models rely on the presence of specific textures when classifying objects. Leveraging TAV, we demonstrate that model accuracy and robustness are heavily influenced by texture. Our results show that texture bias explains the existence of natural adversarial examples, where over 90% of these samples contain textures that are misaligned with the learned texture of their true label, resulting in confident mispredictions.

Problem

• Texture bias in image classification models
• Measuring texture bias with Texture Association Value
• Texture bias explains natural adversarial examples

Innovation

• Introduced Texture Association Value
• Quantifies reliance on specific textures
• Links texture bias to model robustness