Novel Inconsistency Results for Partial Information Decomposition

2025-12-18

AI Summary

Partial information decomposition (PID) aims to quantify redundant, unique, and synergistic contributions of multiple information sources to a target variable, yet has long lacked a uniquely determined, axiomatically consistent solution. Method: We conduct a rigorous axiomatic analysis and formal logical derivation within the PID framework. Contribution/Results: We prove, for the first time, that three foundational principles of classical information theory—non-negativity, the chain rule, and invariance under reversible transformations—are mutually incompatible in PID. This strengthens and generalizes Rauh et al.’s inconsistency result, revealing an unavoidable fundamental trade-off: any well-motivated PID decomposition must abandon at least one of these properties. To guide principled axiom selection, we introduce a mereological modeling framework that provides structured criteria for evaluating axiomatic choices, thereby clarifying the theoretical boundary conditions and practical constraints governing PID design.

Technology Category

• Machine Learning: Information Theory
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Preferences

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Partial Information Decomposition (PID) seeks to disentangle how information about a target variable is distributed across multiple sources, separating redundant, unique, and synergistic contributions. Despite extensive theoretical development and applications across diverse fields, the search for a unique, universally accepted solution remains elusive, with numerous competing proposals offering different decompositions. A promising but underutilized strategy for making progress is to establish inconsistency results, proofs that certain combinations of intuitively appealing axioms cannot be simultaneously satisfied. Such results clarify the landscape of possibilities and force us to recognize where fundamental choices must be made. In this work, we leverage the recently developed mereological approach to PID to establish novel inconsistency results with far-reaching implications. Our main theorem demonstrates that three cornerstone properties of classical information theory, namely non-negativity, the chain rule, and invariance under invertible transformations, become mutually incompatible when extended to the PID setting. This result reveals that any PID framework must sacrifice at least one property that seems fundamental to information theory itself. Additionally, we strengthen the classical result of Rauh et al., which showed that non-negativity, the identity property, and the Williams and Beer axioms cannot coexist.

Problem

• Demonstrates incompatibility of key information theory properties in PID.
• Reveals that any PID framework must sacrifice fundamental information theory principles.
• Strengthens existing inconsistency results regarding PID axioms and properties.

Innovation

• Leverages mereological approach to establish inconsistency results
• Proves classical information theory properties incompatible in PID
• Strengthens prior results on axiom incompatibility in PID