Neural Information Causality

2026-05-10

AI Summary

This work addresses validity and fairness issues in query-separated representation learning arising from information leakage or misestimated channel capacity. To resolve these, the authors propose the Neural Information Causality (Neural-IC) framework, which embeds information causality into representation learning by rigorously disentangling query mechanisms from channel capacity constraints. Through random access communication experiments, information-theoretic inequality analysis, and classical/quantum RAC benchmarks—augmented with CHSH-type correlation layers and conditional information scoring—the framework operationally diagnoses query leakage, precision leakage, and episodic memory effects. Empirical results demonstrate that under hard bit limits, finite precision, or power-constrained channels, Neural-IC accurately identifies representational leakage and reveals that quantum advantage stems from equitable query-conditioned access. Moreover, its multi-depth protocol stability naturally yields the Tsirelson bound.

Abstract

Query-separated computation forces a representation to play an operational role: data are encoded before a query is known, and a later decoder can answer only through the intermediate interface. In this regime the representation functions as a message rather than merely as a feature map. We formalize this observation by embedding information causality (IC) into representation learning, obtaining a framework called neural information causality (Neural-IC). The revised formulation separates two logically distinct statements. First, every query-separated architecture induces a random-access communication experiment and obeys the embedding inequality $I_{\mathrm{N\text{-}RAC}}\le I(\vec a:H,B)$. Second, any independently certified physical capacity bound on the interface, such as a hard $m$-bit alphabet, a finite-precision register, or a power-constrained noisy channel, implies $I_{\mathrm{N\text{-}RAC}}\le C_H$. This separation avoids treating capacity as a post hoc definition and makes Neural-IC an operational diagnostic for query leakage, precision leakage, and episode-specific memory. We also provide an exact one-bit classical RAC benchmark, showing explicitly that the relevant quantum enhancement is not total information beyond the bottleneck, but fair query-conditioned access. For CHSH-type correlation layers, nested Neural-RAC protocols multiply correlation biases across depth; requiring stability of a one-bit bottleneck for arbitrary depth selects the Tsirelson threshold. We extend the analysis to asymmetric seed biases, to multi-capacity finite-depth phase diagrams, and to correlated data via a conditional information score. Controlled simulations, including straight-through binary bottlenecks and deliberately leaky ablations, verify that apparent violations are accounted for by broken query separation or undercounted capacity.

Problem

• information causality
• representation learning
• query separation
• communication bottleneck
• capacity constraint

Innovation

• Neural Information Causality
• query-separated representation
• random-access communication
• information bottleneck
• Tsirelson bound