Neuronal Group Communication for Efficient Neural representation

2025-10-19

AI Summary

Large neural networks struggle to simultaneously achieve representational efficiency, modularity, and interpretability. Method: This paper proposes the Neuron Group Communication (NGC) framework, modeling neural networks as dynamical systems composed of interacting neuron groups, where weights are interpreted as dynamic couplings between neuronal states. A Lyapunov-like stability metric is introduced to theoretically link inference capability with externally induced “potential energy” driving. Training is guided by iterative inter-group communication and low-dimensional signal exchange, explicitly optimizing for system stability. Contribution/Results: NGC significantly improves the performance of large language models on complex reasoning tasks under moderate compression ratios, outperforming conventional low-rank decomposition and cross-layer parameter sharing. Crucially, it establishes the first principled integration of dynamical system stability theory with neuro-symbolic reasoning capabilities—enabling jointly optimized stability and symbolic inference within a unified architectural and training paradigm.

Technology Category

• Machine Learning: Neuro-Symbolic Learning
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

The ever-increasing scale of modern neural networks has brought unprecedented performance alongside daunting challenges in efficiency and interpretability. This paper addresses the core question of how to build large neural systems that learn efficient, modular, and interpretable representations. We propose Neuronal Group Communication (NGC), a theory-driven framework that reimagines a neural network as a dynamical system of interacting neuronal groups rather than a monolithic collection of neural weights. Instead of treating each weight as an independent trainable parameter, NGC treats weights as transient interactions between embedding-like neuronal states, with neural computation unfolding through iterative communication among groups of neurons. This low-rank, modular representation yields compact models: groups of neurons exchange low-dimensional signals, enabling intra-group specialization and inter-group information sharing while dramatically reducing redundant parameters. By drawing on dynamical systems theory, we introduce a neuronal stability metric (analogous to Lyapunov stability) that quantifies the contraction of neuron activations toward stable patterns during sequence processing. Using this metric, we reveal that emergent reasoning capabilities correspond to an external driving force or ``potential'', which nudges the neural dynamics away from trivial trajectories while preserving stability. Empirically, we instantiate NGC in large language models (LLMs) and demonstrate improved performance on complex reasoning benchmarks under moderate compression. NGC consistently outperforms standard low-rank approximations and cross-layer basis-sharing methods at comparable compression rates. We conclude by discussing the broader implications of NGC, including how structured neuronal group dynamics might relate to generalization in high-dimensional learning systems.

Problem

• Developing efficient modular interpretable representations for large neural networks
• Reducing redundant parameters through low-rank neuronal group communication
• Establishing stability metrics linking neural dynamics to reasoning capabilities

Innovation

• Neuronal groups interact dynamically for modular representation
• Low-dimensional signals reduce parameters while enabling specialization
• Stability metric guides emergent reasoning via neural dynamics