How Far Can Sharpness and Complexity Jointly Explain Generalization?

2026-06-27

AI Summary

Existing studies often examine the impact of sharpness or complexity on the generalization of deep neural networks in isolation, overlooking their potential synergistic effects. This work presents the first systematic evaluation of the joint explanatory power of sharpness and complexity, introducing a novel function-space-based metric that relies on weakly dependent parameter representations. The proposed measure is quantitatively validated through linear regression and Pareto front analysis. Experimental results demonstrate that this function-oriented definition substantially enhances the ability to explain generalization behavior, confirming the efficacy of the sharpness–complexity perspective. Nevertheless, the findings also indicate that this combined view, while insightful, remains insufficient as a complete theoretical account of generalization in deep learning.

Abstract

Sharpness and complexity are two central factors in the generalization analysis of deep neural networks. Existing quantitative evaluations of generalization measures have largely focused on individual scalar measures, leaving the joint explanatory power of sharpness and complexity largely unexplored. This work studies how far sharpness and complexity can jointly explain generalization. We use linear regression and introduce a Pareto-based analysis to quantitatively evaluate the joint explanatory power of these two factors. Beyond the existing parameter-level definitions, we further propose realizations of sharpness and complexity that are closer to function space and less dependent on raw parameter representations. We find that function-oriented definitions of these two quantities expand the explanatory scope of the two-factor view beyond what is achieved by existing parameter-level metrics. Overall, our results support the sharpness-complexity perspective as an informative lens for understanding generalization across diverse settings. At the same time, the remaining failures indicate that whether this two-factor view can serve as a complete theory of generalization remains open.

Problem

• sharpness
• complexity
• generalization
• deep neural networks

Innovation

• sharpness
• complexity
• generalization
• function space
• Pareto-based analysis