A scalable estimator of high-order information in complex dynamical systems

2025-06-23

AI Summary

Existing information-theoretic approaches struggle to scalably characterize high-order (≥3rd-order) multivariate dependencies in large-scale neural systems and lack dedicated metrics for multivariate time series. To address this, we propose M-information—a scalable, high-order information integration measure that enables efficient quantification of collective computational mechanisms in macroscopic neural dynamics for the first time. Built upon convex optimization, its robust algorithm exhibits linear computational complexity in the number of variables, supporting analysis of thousand-dimensional neural datasets. M-information is seamlessly embeddable within the information decomposition framework, enabling a principled taxonomy of information dynamics. Experiments demonstrate its strong noise robustness, sensitivity to critical phase transitions, and significant correlation with behavioral performance in mice. Applied to real calcium imaging and electrophysiological data, it successfully uncovers functional synergistic structures beyond pairwise interactions.

Technology Category

• Machine Learning: Information Theory
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: Information Extraction

Application Category

• Web Mining and Content Analysis: Content-based information diffusion
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Our understanding of neural systems rests on our ability to characterise how they perform distributed computation and integrate information. Advances in information theory have introduced several quantities to describe complex information structures, where collective patterns of coordination emerge from high-order (i.e. beyond-pairwise) interdependencies. Unfortunately, the use of these approaches to study large neural systems is severely hindered by the poor scalability of existing techniques. Moreover, there are relatively few measures specifically designed for multivariate time series data. Here we introduce a novel measure of information about macroscopic structures, termed M-information, which quantifies the high-order integration of information in complex dynamical systems. We show that M-information can be calculated via a convex optimisation problem, and we derive a robust and efficient algorithm that scales gracefully with system size. Our analyses show that M-information is resilient to noise, indexes critical behaviour in artificial neuronal populations, and reflects task performance in real-world mouse brain activity data. Furthermore, M-information can be incorporated into existing information decomposition frameworks to reveal a comprehensive taxonomy of information dynamics. Taken together, these results help us unravel collective computation in complex neural systems.

Problem

• Measure high-order information in neural systems
• Scalable analysis for large neural networks
• Quantify information integration in dynamical systems

Innovation

• Introduces M-information for high-order integration
• Uses convex optimization for scalable computation
• Resilient to noise and reflects task performance