On Faster Marginalization with Squared Circuits via Orthonormalization

2024-12-10

arXiv.org

AI Summary

Square circuits suffer from inefficient marginal inference and require explicit normalization in high-dimensional probabilistic modeling. Method: This paper introduces an orthogonal normalization-based parameterization scheme, the first to generalize the canonical form concept from tensor networks to general square circuits. It enables automatic normalization and supports efficient marginal computation over arbitrary variables. Contribution/Results: Theoretically, the method preserves the original expressive power while significantly reducing marginal inference complexity. Empirically, it demonstrates both scalability and accuracy on high-dimensional distribution modeling and inference tasks. This work establishes the first square circuit paradigm that simultaneously guarantees strict normalization, efficient marginal inference, and lossless expressivity—thereby advancing high-dimensional probabilistic reasoning.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Other Foundations of Computer Vision

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Squared tensor networks (TNs) and their generalization as parameterized computational graphs -- squared circuits -- have been recently used as expressive distribution estimators in high dimensions. However, the squaring operation introduces additional complexity when marginalizing variables or computing the partition function, which hinders their usage in machine learning applications. Canonical forms of popular TNs are parameterized via unitary matrices as to simplify the computation of particular marginals, but cannot be mapped to general circuits since these might not correspond to a known TN. Inspired by TN canonical forms, we show how to parameterize squared circuits to ensure they encode already normalized distributions. We then use this parameterization to devise an algorithm to compute any marginal of squared circuits that is more efficient than a previously known one. We conclude by formally showing the proposed parameterization comes with no expressiveness loss for many circuit classes.

Problem

• High-dimensional data distribution
• Computational complexity
• Machine learning application

Innovation

• Parameter Optimization
• Efficient Algorithm
• Circuit Expressivity