Deciphering Shortcut Learning from an Evolutionary Game Theory Perspective

2026-05-04

AI Summary

This work addresses the tendency of deep learning models to rely on spurious shortcut features in data, which undermines generalization and lacks a rigorous theoretical explanation. By leveraging evolutionary game theory, the authors formalize data samples as players and neural tangent features as strategies, thereby defining core and shortcut features in a principled manner. Combining the neural tangent kernel framework with stochastic differential equations, they analyze the dynamics of gradient descent (GD) and stochastic gradient descent (SGD). Theoretically, GD is shown to converge preferentially to shortcut subnetworks, whereas SGD exhibits greater stability in core subnetworks. This analysis elucidates how optimization noise and data noise jointly influence shortcut bias, offering a novel theoretical perspective for mitigating shortcut learning.

Abstract

Shortcut learning causes deep learning models to rely on non-essential features within the data. However, its formation in deep neural network training still lacks theoretical understanding. In this paper, we provide a formal definition of core and shortcut features and employ evolutionary game theory to analyze the origins of shortcut bias by modeling data samples as players and their corresponding neural tangent features as strategies, assuming the existence of core and shortcut subnetworks. We find that gradient descent (GD) and stochastic gradient descent (SGD) lead to two distinct stochastically stable states, each corresponding to a different strategy. The former primarily optimizes the shortcut subnetwork, while the latter primarily optimizes the core subnetwork. We investigate the influence of these strategies on shortcut bias through a continuous stochastic differential equation, and reveal the impact of data noise and optimization noise on the formation of shortcut bias. In brief, our work employs evolutionary game theory to characterize the dynamics of shortcut bias formation and provides a theoretical view on its mitigation.

Problem

• shortcut learning
• deep learning
• evolutionary game theory
• core features
• shortcut bias

Innovation

• evolutionary game theory
• shortcut learning
• neural tangent features
• stochastic gradient descent
• core vs. shortcut features