Explosive neural networks via higher-order interactions in curved statistical manifolds

2024-08-05

arXiv.org

AI Summary

This work addresses two fundamental challenges: the difficulty of modeling higher-order interactions in complex systems and the limited capacity and robustness of conventional associative memory networks. To this end, we propose a novel paradigm—curvature neural networks—grounded in explosive neural dynamics on statistical manifolds of curvature. Our framework integrates the generalized maximum entropy principle, exact mean-field theory, the replica method, and differential-geometric modeling. It is the first to achieve self-regulated annealing and rapid memory retrieval driven by higher-order interactions, analytically revealing explosive phase transitions, multistability, and hysteresis. Near the ferromagnetic–spin-glass phase boundary, the model demonstrates significantly enhanced memory capacity and retrieval robustness, surpassing classical network performance limits. This provides a tractable, analytically solvable theoretical framework for understanding higher-order cooperative mechanisms in both biological and artificial neural systems.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Learning with Manifolds
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Higher-order interactions underlie complex phenomena in systems such as biological and artificial neural networks, but their study is challenging due to the scarcity of tractable models. By leveraging a generalisation of the maximum entropy principle, here we introduce curved neural networks as a class of prototypical models with a limited number of parameters that are particularly well-suited for studying higher-order phenomena. Through exact mean-field descriptions, we show that these curved neural networks implement a self-regulating annealing process that can accelerate memory retrieval, leading to explosive order-disorder phase transitions with multi-stability and hysteresis effects. Moreover, by analytically exploring their memory-retrieval capacity using the replica trick near ferromagnetic and spin-glass phase boundaries, we demonstrate that these networks can enhance memory capacity and robustness of retrieval over classical associative-memory networks. Overall, the proposed framework provides parsimonious models amenable to analytical study, revealing novel higher-order phenomena in complex networks.

Problem

• Model higher-order interactions in neural networks.
• Enhance memory retrieval and network robustness.
• Study explosive phase transitions and multi-stability.

Innovation

• Curved neural networks model
• Self-regulating annealing process
• Enhanced memory retrieval capacity