CoreDPPL: Towards a Sound Composition of Differentiation, ODE Solving, and Probabilistic Programming

2025-03-18

AI Summary

Probabilistic programming, automatic differentiation, and ordinary differential equation (ODE) modeling have long been studied in isolation, leading to unsafe compositions and a lack of unified semantic foundations. Method: We introduce the first core language supporting higher-order probabilistic programming, first-class differentiation operators, and rigorous ODE semantics. Its dependently typed static type system formally separates stochastic computation from deterministic differentiation, preventing unsafe combinations. We mechanize the full formal semantics and type safety proof in Agda. Contribution/Results: The resulting language—DPPL—achieves end-to-end type safety; all key metatheoretic properties—including progress and preservation—are machine-checked. This establishes the first compositional safety framework unifying probabilistic reasoning, differentiability, and continuous dynamical modeling, providing a formal foundation for trustworthy scientific computing and AI-driven modeling.

Technology Category

• Reasoning under Uncertainty: Probabilistic Programming
• Natural Language Processing: Safety and Robustness
• Machine Learning: Probabilistic Circuits and Graphical Models

Application Category

• Security and Privacy: Data transparency and provenance
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

In recent years, there has been extensive research on how to extend general-purpose programming language semantics with domain-specific modeling constructs. Two areas of particular interest are (i) universal probabilistic programming where Bayesian probabilistic models are encoded as programs, and (ii) differentiable programming where differentiation operators are first class or differential equations are part of the language semantics. These kinds of languages and their language constructs are usually studied separately or composed in restrictive ways. In this paper, we study and formalize the combination of probabilistic programming constructs, first-class differentiation, and ordinary differential equations in a higher-order setting. We propose formal semantics for a core of such differentiable probabilistic programming language (DPPL), where the type system tracks random computations and rejects unsafe compositions during type checking. The semantics and its type system are formalized, mechanized, and proven sound in Agda with respect to abstract language constructs.

Problem

• Combining probabilistic programming with differentiation and ODEs
• Formalizing semantics for a differentiable probabilistic programming language
• Ensuring type safety in random computations and language constructs

Innovation

• Combines probabilistic programming with differentiation
• Integrates ODE solving in higher-order language
• Uses type system to ensure safe compositions