Scaling Optimization Over Uncertainty via Compilation

2025-02-26

AI Summary

This work addresses the scalability challenge of optimization tasks in probabilistic inference—specifically, maximum expected utility (MEU) and nested marginal MAP (MMAP)—under uncertainty. Methodologically, it introduces an “optimization-as-compilation” paradigm, centered on a novel branch-and-bound semiring and its weighted binary decision diagram (BDD) representation, enabling structure-aware problem decomposition and suboptimal solution pruning; it further incorporates a staged compilation mechanism that supports query-driven, incremental optimization at runtime. Based on this foundation, the authors implement two probabilistic programming languages: the functional DAPPL and the imperative PINEAPPL—marking the first unified framework supporting both exact probabilistic inference and complex decision optimization. Experimental evaluation demonstrates substantial improvements in optimization scale and flexibility, establishing a scalable, formally verifiable approach to decision optimization under uncertainty.

Technology Category

• Reasoning under Uncertainty: Probabilistic Programming
• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Constraint Programming

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Probabilistic inference is fundamentally hard, yet many tasks require optimization on top of inference, which is even harder. We present a new extit{optimization-via-compilation} strategy to scalably solve a certain class of such problems. In particular, we introduce a new intermediate representation (IR), binary decision diagrams weighted by a novel notion of extit{branch-and-bound semiring}, that enables a scalable branch-and-bound based optimization procedure. This IR automatically extit{factorizes} problems through program structure and extit{prunes} suboptimal values via a straightforward branch-and-bound style algorithm to find optima. Additionally, the IR is naturally amenable to extit{staged compilation}, allowing the programmer to query for optima mid-compilation to inform further executions of the program. We showcase the effectiveness and flexibility of the IR by implementing two performant languages that both compile to it: extsc{dappl} and extsc{pineappl}. extsc{dappl} is a functional language that solves maximum expected utility problems with first-class support for rewards, decision making, and conditioning. extsc{pineappl} is an imperative language that performs exact probabilistic inference with support for nested marginal maximum a posteriori (MMAP) optimization via staging.

Problem

• Scalable optimization over probabilistic inference uncertainty
• Compilation strategy for branch-and-bound based optimization
• Intermediate representation enabling staged compilation for optima queries

Innovation

• New intermediate representation with branch-and-bound semiring
• Scalable branch-and-bound optimization via program factorization
• Staged compilation enabling mid-compilation optima queries