The Representational Limit of Scalar Interactions: An Interventional Decomposition

2026-06-17

AI Summary

Existing scalar pairwise interaction measures struggle to disentangle the unique, redundant, and synergistic contributions among features, often leading to misinterpretations of model decision mechanisms. This work proposes Stochastic Hi-Fi, the first method to achieve a predictability decomposition with rigorous interventional semantics that cleanly separates unique (U), redundant (R), and synergistic (S) interactions—without requiring model retraining. By integrating interventional masking inference, coupled diamond sampling, and Monte Carlo estimation within a finite vocabulary space, the approach delivers variance reduction, finite-sample error bounds, and uniform convergence guarantees. Empirically, it accelerates interaction recovery by 411× in structural causal models and successfully disentangles redundant and synergistic attention heads in GPT-2. On the NIH ChestX-ray14 dataset, feature ablation guided by Stochastic Hi-Fi yields significantly higher AUC degradation than GradCAM.

Abstract

Signed pairwise interaction scores fundamentally conflate uniqueness (U), redundancy (R), and synergy (S). We prove this on a minimal 3-way XOR structural causal model: faithful indices such as Shapley-Taylor return zero per pair, whereas projective indices such as Shapley Interaction spread the third-order effect into pair scalars that conflate the three mechanisms. We introduce Stochastic Hi-Fi, a post-hoc, retraining-free predictability decomposition that estimates per-feature U/R/S profiles by interventional masked inference. The estimator provides exact interventional semantics, finite-sample Monte Carlo bounds, strict variance reduction from coupled diamond sampling, and uniform finite-vocabulary convergence. Across tabular SCMs, Stochastic Hi-Fi recovers structure missed by scalar baselines (up to 411x larger interaction-magnitude recovery ratios). It also separates redundant and synergistic heads in the GPT-2 IOI circuit. On NIH ChestX-ray14, Stochastic Hi-Fi matches GradCAM on Pointing Game and improves substantially on Deletion AUC.

Problem

• scalar interactions
• uniqueness
• redundancy
• synergy
• interaction decomposition

Innovation

• Stochastic Hi-Fi
• interventional decomposition
• synergy
• redundancy
• masked inference