On the Relationship Between Monotone and Squared Probabilistic Circuits

2024-08-01

arXiv.org

AI Summary

Monotonic probabilistic circuits (PCs) and squared PCs represent two distinct modeling paradigms whose expressive powers are incomparable, hindering joint optimization and unified analysis. Method: We propose Inception PCs—a novel PC architecture integrating structural heterogeneity and complex-valued parameters—that strictly generalizes both monotonic and squared PCs, enabling differentiable, unified density estimation. Our approach jointly incorporates complex-weight optimization, differentiable structure learning, and guaranteed tractable marginal inference. Contribution/Results: We provide the first rigorous proof that monotonic and squared PCs are mutually non-dominating in expressivity. Empirically, Inception PCs achieve state-of-the-art performance across multiple tabular and image benchmarks, significantly outperforming baselines in log-likelihood estimation and generalization—demonstrating superior modeling capacity, optimization stability, and inference tractability.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

Probabilistic circuits are a unifying representation of functions as computation graphs of weighted sums and products. Their primary application is in probabilistic modeling, where circuits with non-negative weights (monotone circuits) can be used to represent and learn density/mass functions, with tractable marginal inference. Recently, it was proposed to instead represent densities as the square of the circuit function (squared circuits); this allows the use of negative weights while retaining tractability, and can be exponentially more expressive efficient than monotone circuits. Unfortunately, we show the reverse also holds, meaning that monotone circuits and squared circuits are incomparable in general. This raises the question of whether we can reconcile, and indeed improve upon the two modeling approaches. We answer in the positive by proposing Inception PCs, a novel type of circuit that naturally encompasses both monotone circuits and squared circuits as special cases, and employs complex parameters. Empirically, we validate that Inception PCs can outperform both monotone and squared circuits on a range of tabular and image datasets.

Problem

• Monotone vs. squared circuit expressiveness
• Reconciling monotone and squared circuit approaches
• Proposing Inception PCs for improved modeling

Innovation

• Inception PCs with complex parameters
• Combines monotone and squared circuits
• Outperforms on tabular, image datasets