Measuring the Effect of Background on Classification and Feature Importance in Deep Learning for AV Perception

2025-12-05

AI Summary

This work addresses two critical challenges in autonomous driving perception: (i) the implicit influence of background features on deep learning classification decisions, and (ii) the difficulty of quantifying the reliability of eXplainable AI (XAI) explanations. To this end, we propose the first controlled-variable framework for assessing background importance. We construct six synthetic datasets to systematically decouple three factors—background correlation, camera viewpoint variation, and traffic sign shape—and employ saliency-based XAI methods (e.g., SHAP, GradCAM) alongside mask-based annotations to quantify their independent effects on classification performance and background dependency. Experiments reveal that background importance increases significantly under training-domain shift and exhibits a strong negative correlation with model generalization. Camera viewpoint variation exacerbates background dependency, whereas high shape specificity mitigates it. Our framework provides a reproducible, quantitative paradigm for enhancing model robustness and XAI trustworthiness in safety-critical perception tasks.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Calibration & Uncertainty Quantification
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Common approaches to explainable AI (XAI) for deep learning focus on analyzing the importance of input features on the classification task in a given model: saliency methods like SHAP and GradCAM are used to measure the impact of spatial regions of the input image on the classification result. Combined with ground truth information about the location of the object in the input image (e.g., a binary mask), it is determined whether object pixels had a high impact on the classification result, or whether the classification focused on background pixels. The former is considered to be a sign of a healthy classifier, whereas the latter is assumed to suggest overfitting on spurious correlations. A major challenge, however, is that these intuitive interpretations are difficult to test quantitatively, and hence the output of such explanations lacks an explanation itself. One particular reason is that correlations in real-world data are difficult to avoid, and whether they are spurious or legitimate is debatable. Synthetic data in turn can facilitate to actively enable or disable correlations where desired but often lack a sufficient quantification of realism and stochastic properties. [...] Therefore, we systematically generate six synthetic datasets for the task of traffic sign recognition, which differ only in their degree of camera variation and background correlation [...] to quantify the isolated influence of background correlation, different levels of camera variation, and considered traffic sign shapes on the classification performance, as well as background feature importance. [...] Results include a quantification of when and how much background features gain importance to support the classification task based on changes in the training domain [...]. Download: synset.de/datasets/synset-signset-ger/background-effect

Problem

• Quantifies background correlation impact on deep learning classification performance.
• Systematically generates synthetic datasets to isolate and measure background influence.
• Evaluates when background features gain importance in traffic sign recognition tasks.

Innovation

• Synthetic datasets isolate background correlation effects
• Quantify background feature importance in classification tasks
• Systematically vary camera and background in traffic sign recognition