The Quadrilateral Loss: Additivity as a Measurable Behavior of Dense Neural Networks
This work proposes a novel “quadrilateral loss” to achieve interpretable additive behavior while preserving model performance and avoiding the black-box nature induced by feature interactions. Unlike prior approaches that enforce additivity through structural constraints, this method formulates additivity as a continuously tunable behavioral objective. It introduces a differentiable penalty based on second-order mixed differences to suppress unwanted interactions and defines an online-observable interaction strength metric, revealing the unreliability of posterior interaction rankings. By integrating intervention-based Shapley-GAM, structural masking, and behavioral regularization, the approach simultaneously enhances both accuracy and additivity with only mild penalties on small datasets. Experiments demonstrate convergence of shape functions across diverse additive pathways and show that behavioral constraints substantially outperform conventional weight-space regularization.