equivariant networks

This paper addresses the unified modeling of symmetries in machine learning. It proposes a framework grounded in differential geometry and convex optimization to (1) enforce known symmetries, (2) automatically discover unknown symmetries in models or data, and (3) actively induce symmetry breaking via user-specified candidate groups. The core contribution is the first formulation of symmetry imposition and discovery as dual linear-algebraic tasks, leveraging the Lie derivative to characterize fiberwise linear Lie group actions on vector bundles, and employing nuclear-norm relaxation to construct convex regularization terms. The method is broadly applicable to neural networks, dynamical system discovery, basis-function regression, and neural operators. Empirically, it significantly improves generalization performance and parameter efficiency—particularly in low-data regimes—while preserving geometric structure and interpretability.

Representation learning in artificial systems suffers from a fundamental lack of invariance and equivariance, limiting generalization and compositional reasoning. Method: We propose the Symmetry-Loss principle—a differentiable group-constrained optimization framework grounded in environmental symmetry priors. It models representation learning as a progressive developmental refinement of effective symmetry groups, driven by minimization of structural surprise (a free-energy-like objective). The approach integrates Lie group priors, differentiable symmetry constraints, free-energy minimization, and prediction-error optimization. Contribution/Results: This work unifies cortical development mechanisms, predictive coding, and group theory under a novel free-energy principle for symmetry self-organization. Experiments demonstrate that learned representations exhibit high efficiency, robustness to perturbations, and compositional structure. The framework provides a biologically plausible, brain-inspired theoretical foundation for both cortical development modeling and embodied intelligence.

This paper investigates whether deep networks under standard supervised training can autonomously learn unencoded symmetries—such as rotation invariance—from partially observed cyclic group-symmetric data, under realistic class-level symmetry heterogeneity (where some classes exhibit full cyclic transformations while others only subsets). Method: We develop the first neural-kernel symmetry learning theory, grounded in infinite-width NTK analysis and group representation-theoretic Fourier analysis. Contribution/Results: The theory reveals that generalization to unseen symmetries hinges on the “overwhelming dominance” of local data structure over symmetry structure within the kernel-induced feature space, and yields a verifiable signal-to-noise criterion in the frequency domain. Our analysis precisely reproduces empirical failures of MLPs, CNNs, and ViTs on rotationally augmented MNIST subsets, and rigorously proves that conventional supervised training cannot acquire symmetries absent from the architectural prior.

Equivariant networks strictly preserve input symmetries, rendering them ill-suited for generative tasks requiring *active symmetry breaking*—e.g., reconstructing asymmetric structures from highly symmetric latent representations. To address this, we establish the first necessary and sufficient representation theorem for equivariant conditional distributions and propose SymPE: a method that achieves *controllable symmetry breaking* via learnable stochastic normalized positional encodings, while preserving the group-equivariant inductive bias. SymPE unifies probabilistic symmetry breaking, positional encoding, and equivariant graph neural networks, and naturally integrates with diffusion-based generative frameworks. Empirically, it significantly improves performance on graph diffusion modeling, graph autoencoding, and lattice spin system generation. Theoretically, we prove that SymPE’s generalization bound is strictly superior to that of conventional equivariant networks.

This work investigates how task-relevant symmetries—exact or approximate equivariance—affect the generalization of deep learning models, particularly under symmetry mismatch between model and data. Method: We develop the first generalization bound that does not assume group structure, rigorously quantifying the interplay between model equivariance error and data equivariance error. Our approach integrates probabilistic generalization theory, function approximation theory, and symmetry metrics, accommodating non-group, non-exact, and non-global equivariance settings. Contributions/Results: We establish that precise modeling of task symmetries significantly improves generalization. We formally characterize the optimal error trade-off under approximate or local equivariance when model and data symmetries are misaligned. Furthermore, we derive an “error alignment” principle—a concrete, actionable theoretical guideline for designing robust equivariant models—thereby bridging abstract symmetry considerations with practical architectural design.

Existing equivariant networks typically support only fixed symmetry groups, limiting their ability to flexibly handle multimodal data with diverse symmetries. This work proposes the ASEN model, which achieves simultaneous equivariance to multiple permutation subgroups within a single architecture by incorporating symmetry-breaking auxiliary input features and leveraging an approximate symmetry-breaking mechanism together with an efficient 2-closure fast algorithm. Built upon a fully permutation-equivariant basis model and employing equivariant MLP emulation techniques, ASEN overcomes the rigidity of conventional equivariant networks. Experiments demonstrate that ASEN outperforms both specialized equivariant models and non-equivariant baselines across tasks involving graph and image symmetry selection, as well as sequence-based multitask and transfer learning scenarios.

This work addresses the issue of symmetry amplification in equivariant neural networks, which can impair their ability to represent geometric structures when processing symmetric inputs. The study provides the first rigorous proof that this phenomenon is bounded below by a limit determined by the structure of the feature space. Leveraging group representation theory and an analysis of equivariant mappings, the authors develop a computable framework to determine this infimum. Based on this theoretical foundation, they propose design principles for constructing features that mitigate detrimental symmetry amplification. Experiments on synthetic data and the QM9 dataset validate the theoretical predictions, demonstrating that the proposed approach effectively alleviates symmetry amplification and enhances model expressivity.

This work investigates how to automatically uncover low-dimensional constraint structures induced by symmetries and conservation laws from high-dimensional physical data in the absence of explicit prior knowledge. To this end, we propose an unsupervised representation learning framework based on variational autoencoders that eschews conventional designs relying on explicit symmetry embeddings. Instead, our approach leverages the information bottleneck principle to drive the latent space to self-organize and reveal the dimensionality reduction inherent to underlying symmetries. Evaluated on geometric systems and particle physics datasets, the method successfully recovers theoretically expected symmetry structures and systematically delineates the theoretical limits and practical challenges of symmetry inference under minimal inductive bias.

Existing equivariant neural fields struggle to handle inconsistent group actions on heterogeneous product spaces. This work proposes an isotropy subgroup reduction framework that establishes an orbit equivalence $(X \times M)/G \cong X/H$, thereby transforming the learning of $G$-invariant functions over the product space into learning $H$-invariant functions solely on $X$, where $H$ is the isotropy subgroup. By circumventing the stringent structural constraints on group actions imposed by prior methods, this approach significantly enhances modeling flexibility while preserving expressive capacity. It achieves, for the first time, a unified equivariant modeling framework applicable to arbitrary group actions and homogeneous configuration spaces.

This work addresses the underexplored role of optimizers in training equivariant and geometric neural networks, which, despite their ability to encode geometric symmetries, often underperform unconstrained models due to optimization challenges. For the first time, we systematically compare the Muon and Adam optimizers across a range of equivariant architectures, employing Hessian curvature estimation, loss landscape visualization, and spectral analysis of weights—including stable and effective ranks—to reveal how optimizer choice profoundly shapes training dynamics and representational properties. On ModelNet40 point cloud classification and molecular tasks, Muon consistently outperforms Adam across all architectures, yielding models with smoother loss landscapes, higher curvature, and higher-rank weight matrices and intermediate features, thereby highlighting the critical interplay between optimizer design and geometric inductive biases.