Shortcut Mitigation via Spurious-Positive Samples

2026-05-13

AI Summary

Existing shortcut learning mitigation methods typically rely on fully annotated data, group-balanced validation sets, or training data that exhaustively covers all attribute–class combinations—conditions rarely met in real-world scenarios. This work proposes a novel approach that requires neither additional annotations nor balanced validation sets. By analyzing internal model representations, the method identifies a small set of samples exhibiting spurious correlations and locates critical neurons responsible for leveraging these misleading attributes, guided by the principle that such features should not inform predictions. Intermediate-layer regularization is then applied to these neurons. Requiring only a few spuriously correlated (false positive) samples, the approach effectively suppresses shortcut learning, significantly enhancing model robustness and preventing models from making correct predictions for incorrect reasons, thereby achieving effective mitigation under more realistic data conditions.

Abstract

Shortcut mitigation strategies commonly rely on training data annotations, group-balanced held-out data or the presence of all groups, i.e., all combinations of (spurious) attributes and classes, in the training data. However, these requirements are rarely met in practice. We instead propose a method for targeted model analysis to identify a small set of instances in which the model relies on spurious attributes. Using that set and following ``this feature should not be used for prediction'' reasoning, we identify highly relevant neurons in an intermediate layer and regularize their impact. This ensures that models learn to depend on informative features rather than being right for the wrong reasons, thereby improving robustness without requiring additional balanced held-out data or annotations.

Problem

• shortcut learning
• spurious correlations
• model robustness
• data imbalance
• feature dependence

Innovation

• shortcut mitigation
• spurious-positive samples
• neuron regularization
• model robustness
• feature attribution