An XAI-based Analysis of Shortcut Learning in Neural Networks

2025-04-22

AI Summary

Neural networks often rely on spurious features—statistically correlated with labels but lacking causal relevance—leading to unstable performance of existing mitigation methods. This paper systematically investigates the encoding mechanisms of spurious correlations and introduces an XAI-based Neuron Spuriousness Score (NSS), the first metric enabling cross-architectural (CNN/ViT) and neuron-level quantification of spurious feature dependence. We find that spurious features are partially disentangled within models, with disentanglement degree critically dependent on architecture; moreover, foundational assumptions underlying current debiasing methods are fundamentally incomplete. Our work provides a novel diagnostic tool and theoretical foundation for modeling spurious correlations, advancing the development of more robust, interpretable, and secure AI systems.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

Machine learning models tend to learn spurious features - features that strongly correlate with target labels but are not causal. Existing approaches to mitigate models' dependence on spurious features work in some cases, but fail in others. In this paper, we systematically analyze how and where neural networks encode spurious correlations. We introduce the neuron spurious score, an XAI-based diagnostic measure to quantify a neuron's dependence on spurious features. We analyze both convolutional neural networks (CNNs) and vision transformers (ViTs) using architecture-specific methods. Our results show that spurious features are partially disentangled, but the degree of disentanglement varies across model architectures. Furthermore, we find that the assumptions behind existing mitigation methods are incomplete. Our results lay the groundwork for the development of novel methods to mitigate spurious correlations and make AI models safer to use in practice.

Problem

• Analyzing how neural networks encode spurious correlations
• Measuring neuron dependence on spurious features via XAI
• Evaluating mitigation methods for spurious feature reliance

Innovation

• XAI-based neuron spurious score for diagnostics
• Analyzes CNNs and ViTs with architecture-specific methods
• Identifies incomplete assumptions in existing mitigation methods