Restructuring Tractable Probabilistic Circuits

2024-11-19

arXiv.org

AI Summary

Probabilistic circuits (PCs) lack efficient multiplication across distinct variable trees (v-trees), hindering compositional inference and limiting practical deployment. Method: We propose a general, lossless structural PC recompilation framework that converts any structured PC into a target v-tree in polynomial time. Our approach integrates three core techniques: (i) v-tree–guided structural transformation via variable remapping, (ii) node reorganization preserving structural decomposability, and (iii) a depth-reduction algorithm optimized via dynamic range decomposition. Contribution/Results: This is the first method enabling polynomial-time PC multiplication across arbitrary v-trees, fully decoupling structural management during training from inference-time composition. Experiments demonstrate substantial speedups in heterogeneous PC multiplication and enable real-time, controllable inference in applications such as text generation. The framework establishes a new foundation for scalable, trainable PC architectures.

Technology Category

• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Probabilistic circuits (PCs) are a unifying representation for probabilistic models that support tractable inference. Numerous applications of PCs like controllable text generation depend on the ability to efficiently multiply two circuits. Existing multiplication algorithms require that the circuits respect the same structure, i.e. variable scopes decomposes according to the same vtree. In this work, we propose and study the task of restructuring structured(-decomposable) PCs, that is, transforming a structured PC such that it conforms to a target vtree. We propose a generic approach for this problem and show that it leads to novel polynomial-time algorithms for multiplying circuits respecting different vtrees, as well as a practical depth-reduction algorithm that preserves structured decomposibility. Our work opens up new avenues for tractable PC inference, suggesting the possibility of training with less restrictive PC structures while enabling efficient inference by changing their structures at inference time.

Problem

• Restructuring probabilistic circuits to conform to target vtrees
• Enabling multiplication of circuits with different vtree structures
• Providing depth-reduction while preserving structured decomposability

Innovation

• Restructuring PCs to conform to target vtrees
• Novel polynomial-time multiplication for different vtrees
• Depth-reduction preserving structured decomposibility