Functional Information Decomposition: A First-Principles Approach to Analyzing Functional Relationships

2025-09-22

AI Summary

This work addresses the challenge of decomposing multivariate information contributions to a target variable in complex systems. We propose the Functional Information Decomposition (FID) framework, which departs from conventional statistical correlation–based mutual information decomposition by rigorously grounding the analysis in complete deterministic functional relationships between inputs and outputs. Derived from first principles and integrated with multivariate mutual information theory, FID enables precise quantification of unique, redundant, and synergistic information contributions. Crucially, FID is the first framework to model information decomposition explicitly atop functional dependency structures—thereby overcoming the limitations of approaches relying solely on joint distributions. Empirical evaluations demonstrate that FID effectively disentangles the functional information roles of variables in intricate systems, substantially enhancing interpretability. As such, it provides a novel theoretical foundation for applications in neuroscience, causal inference, and complex systems analysis.

Technology Category

• Machine Learning: Information Theory
• Reasoning under Uncertainty: Relational Probabilistic Models
• Natural Language Processing: Information Extraction

Application Category

• Web Mining and Content Analysis: Content-based information diffusion
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

Information theory, originating from Shannon's work on communication systems, has become a fundamental tool across neuroscience, genetics, physics, and machine learning. However, the application of information theory is often limited to the simplest case: mutual information between two variables. A central challenge in extending information theory to multivariate systems is decomposition: understanding how the information that multiple variables collectively provide about a target can be broken down into the distinct contributions that are assignable to individual variables or their interactions. To restate the problem clearly, what is sought after is a decomposition of the mutual information between a set of inputs (or parts) and an output (or whole). In this work, we introduce Functional Information Decomposition (FID) a new approach to information decomposition that differs from prior methods by operating on complete functional relationships rather than statistical correlations, enabling precise quantification of independent and synergistic contributions.

Problem

• Decomposing multivariate information into individual and interactive contributions
• Extending information theory beyond simple two-variable mutual information
• Quantifying independent and synergistic information in functional relationships

Innovation

• Decomposes mutual information using functional relationships
• Quantifies independent and synergistic information contributions
• Operates on complete functional rather than statistical correlations