A Compositional Theory of Curvature in Probabilistic Circuits
This work addresses the limitations of global sharpness-aware regularization in probabilistic circuits, which induces depth bias and underfitting by disregarding the compositional curvature structure of the loss landscape. The study reveals, for the first time, that the trace of the Hessian in probabilistic circuits admits a decomposable form, precisely factorizing into the product of circuit flows and local sharpness terms. Building on this insight, the authors propose an adaptive sharpness-aware regularization method grounded in local intrinsic curvature. This approach preserves the closed-form updates of the EM algorithm while effectively balancing model generalization and training stability. Empirical results demonstrate that the proposed method substantially recovers the generalization performance sacrificed by global regularization, without compromising the robustness inherent to sharpness-aware learning.