Conditional diffusions for neural posterior estimation

2024-10-24

arXiv.org

AI Summary

Normalizing flow-based neural posterior estimation (NPE) under black-box likelihoods suffers from training instability and a fundamental trade-off between expressive power and computational cost. Method: This paper introduces conditional diffusion models into the NPE framework for the first time, proposing a denoising diffusion probabilistic model (DDPM)-based conditional NPE method. Unlike flow-based approaches, it imposes no invertibility constraints and natively supports both summary networks and end-to-end modeling. Contribution/Results: Experiments across diverse benchmark tasks demonstrate that the proposed method significantly improves posterior estimation accuracy and training stability, achieves faster convergence, and outperforms state-of-the-art normalizing flow models—even when using shallower architectures. It establishes a more robust and computationally efficient paradigm for simulation-based Bayesian inference.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Neural posterior estimation (NPE), a simulation-based computational approach for Bayesian inference, has shown great success in situations where posteriors are intractable or likelihood functions are treated as"black boxes."Existing NPE methods typically rely on normalizing flows, which transform a base distributions into a complex posterior by composing many simple, invertible transformations. But flow-based models, while state of the art for NPE, are known to suffer from several limitations, including training instability and sharp trade-offs between representational power and computational cost. In this work, we demonstrate the effectiveness of conditional diffusions as an alternative to normalizing flows for NPE. Conditional diffusions address many of the challenges faced by flow-based methods. Our results show that, across a highly varied suite of benchmarking problems for NPE architectures, diffusions offer improved stability, superior accuracy, and faster training times, even with simpler, shallower models. These gains persist across a variety of different encoder or"summary network"architectures, as well as in situations where no summary network is required. The code will be publicly available at url{https://github.com/TianyuCodings/cDiff}.

Problem

• Improving neural posterior estimation stability and accuracy
• Overcoming limitations of flow-based models in NPE
• Enhancing training efficiency with conditional diffusions

Innovation

• Conditional diffusions enhance neural posterior estimation
• High-capacity summary networks improve accuracy and stability
• Simpler models achieve faster training and superior performance