Implicit Bias and Invariance: How Hopfield Networks Efficiently Learn Graph Orbits

2025-12-16

AI Summary

This work investigates how Hopfield networks implicitly learn invariant representations of graph isomorphism classes from few random graph samples. Methodologically, we first prove that any graph isomorphism class can be embedded into a three-dimensional invariant subspace under the action of the permutation group; we then introduce Minimum Energy Flow (MEF) gradient descent, revealing its implicit bias toward norm efficiency, and derive a polynomial upper bound on sample complexity based on this bias. Theoretically, network parameters asymptotically converge to this invariant subspace as sample size increases; empirically, the mechanism enables efficient isomorphism class inference and strong generalization. Our core contributions are: (i) establishing the first theoretical framework for implicit invariant learning in Hopfield networks; (ii) uncovering an intrinsic unification between implicit bias—specifically norm-efficient optimization—and group invariance; and (iii) providing a novel principle for few-shot learning on graph-structured data.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Learning to Search
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

Many learning problems involve symmetries, and while invariance can be built into neural architectures, it can also emerge implicitly when training on group-structured data. We study this phenomenon in classical Hopfield networks and show they can infer the full isomorphism class of a graph from a small random sample. Our results reveal that: (i) graph isomorphism classes can be represented within a three-dimensional invariant subspace, (ii) using gradient descent to minimize energy flow (MEF) has an implicit bias toward norm-efficient solutions, which underpins a polynomial sample complexity bound for learning isomorphism classes, and (iii) across multiple learning rules, parameters converge toward the invariant subspace as sample sizes grow. Together, these findings highlight a unifying mechanism for generalization in Hopfield networks: a bias toward norm efficiency in learning drives the emergence of approximate invariance under group-structured data.

Problem

• Studies implicit bias and invariance in Hopfield networks learning graph isomorphism classes.
• Analyzes how gradient descent minimizes energy flow for norm-efficient solutions.
• Explores convergence toward invariant subspaces with increasing sample sizes.

Innovation

• Hopfield networks learn graph isomorphism via implicit bias
• Gradient descent minimizes energy flow for norm-efficient solutions
• Parameters converge to invariant subspace with increasing samples