On Symmetries in Convolutional Weights

2025-03-24

AI Summary

Convolutional neural networks (CNNs) exhibit unexplained geometric generalization capabilities, yet the underlying structural priors remain poorly understood. Method: This work systematically analyzes average $k imes k$ weight kernels across layers of diverse CNNs (e.g., ResNet, VGG) trained on multiple datasets (CIFAR, ImageNet), employing statistical quantification, visualization, and controlled ablation studies to isolate architectural influences. Contribution/Results: We discover a robust, architecture-induced center symmetry in layer-wise average kernels—arising intrinsically from convolutional operations—and establish it as a fundamental inductive bias of CNNs. Crucially, this symmetry strongly correlates with improved generalization under geometric transformations: models with higher kernel symmetry demonstrate superior translational and reflective consistency. To our knowledge, this is the first work to formally identify average kernel symmetry as an inherent structural prior, offering a novel perspective on CNNs’ implicit regularization and geometric generalization mechanisms.

Technology Category

• Computer Vision: Other Foundations of Computer Vision
• Machine Learning: Deep Neural Architectures and Foundation Models
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Responsible Web: Measurement, analysis, and circumvention of Web censorship

Abstract

We explore the symmetry of the mean k x k weight kernel in each layer of various convolutional neural networks. Unlike individual neurons, the mean kernels in internal layers tend to be symmetric about their centers instead of favoring specific directions. We investigate why this symmetry emerges in various datasets and models, and how it is impacted by certain architectural choices. We show how symmetry correlates with desirable properties such as shift and flip consistency, and might constitute an inherent inductive bias in convolutional neural networks.

Problem

• Explores symmetry in mean convolutional weight kernels
• Investigates causes of symmetry in various models
• Links symmetry to desirable network properties

Innovation

• Explores mean kernel symmetry in CNNs
• Links symmetry to shift consistency
• Investigates symmetry's inductive bias role