Revisiting Generalization Power of a DNN in Terms of Symbolic Interactions

2025-02-14

AI Summary

Understanding the generalization mechanisms of deep neural networks (DNNs) remains a fundamental challenge in deep learning theory. Method: This work investigates DNN generalization from an interaction perspective, proposing a “symbolic interaction” modeling framework. It theoretically derives a rigorous decoupling of generalizable versus non-generalizable interactions and establishes an interpretable generalization attribution framework grounded in interaction distribution morphology—revealing that generalizable interactions follow a decaying distribution, whereas non-generalizable ones exhibit a spindle-shaped distribution. Contribution/Results: Empirical measurements across diverse models and datasets confirm strong alignment between theoretical predictions and actual internal interactions in trained DNNs. The proposed distribution-based criterion successfully isolates generalization-relevant interaction components, validating the universality of the observed distributional规律. This work provides both a novel conceptual lens for understanding DNN generalization and a practical, interpretable analytical tool for diagnosing and improving model generalizability.

Technology Category

• Machine Learning: Deep Learning Theory
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

This paper aims to analyze the generalization power of deep neural networks (DNNs) from the perspective of interactions. Unlike previous analysis of a DNN's generalization power in a highdimensional feature space, we find that the generalization power of a DNN can be explained as the generalization power of the interactions. We found that the generalizable interactions follow a decay-shaped distribution, while non-generalizable interactions follow a spindle-shaped distribution. Furthermore, our theory can effectively disentangle these two types of interactions from a DNN. We have verified that our theory can well match real interactions in a DNN in experiments.

Problem

• Analyze DNN generalization via symbolic interactions
• Distinguish generalizable and non-generalizable interaction distributions
• Disentangle interaction types in deep neural networks

Innovation

• Analyzes DNN generalization via interactions
• Distinguishes decay-shaped generalizable interactions
• Disentangles generalizable and non-generalizable interactions