🤖 AI Summary
This work proposes a novel “quadrilateral loss” to achieve interpretable additive behavior while preserving model performance and avoiding the black-box nature induced by feature interactions. Unlike prior approaches that enforce additivity through structural constraints, this method formulates additivity as a continuously tunable behavioral objective. It introduces a differentiable penalty based on second-order mixed differences to suppress unwanted interactions and defines an online-observable interaction strength metric, revealing the unreliability of posterior interaction rankings. By integrating intervention-based Shapley-GAM, structural masking, and behavioral regularization, the approach simultaneously enhances both accuracy and additivity with only mild penalties on small datasets. Experiments demonstrate convergence of shape functions across diverse additive pathways and show that behavioral constraints substantially outperform conventional weight-space regularization.
📝 Abstract
Additive models buy interpretability by forbidding feature interactions, a constraint that neural instantiations enforce architecturally. We introduce the quadrilateral loss, a differentiable penalty that treats additivity as a measurable behavior instead: a second-order mixed difference on pairs of training points swapping one coordinate, which vanishes if and only if the coordinate carries no interaction, remains informative for piecewise-linear networks, and equals in expectation the per-coordinate interaction mass of the interventional Shapley-GAM. The loss turns additivity into a dial - most learned interactions prove removable almost for free, and on small datasets a moderate penalty improves accuracy and additivity simultaneously - and into an online observable: its per-feature surrender curves show, across seeds and datasets, that pre-regularization interaction magnitude barely predicts what a regularized model retains, undermining post-hoc interaction rankings. Against this instrument we compare routes to exact additivity, spanning structural masks, behavioral penalties (optionally crystallized into exact structure), weight decay, backfitting, the shared-section model, and bagged boosted stumps: constraining behavior before structure dominates weight-space constraints, rankings reverse between data regimes, and converging routes agree on the shape functions themselves. Three silent failure modes we document share one anatomy: guarantees imported into settings that quietly void their preconditions.