Simulation-based Inference via Langevin Dynamics with Score Matching

2025-09-03

AI Summary

Bayesian inference with intractable likelihoods but tractable simulators remains challenging, particularly in medium- to high-dimensional parameter spaces where conventional simulation-based inference (SBI) methods—such as approximate Bayesian computation (ABC) and deep generative models—suffer from poor efficiency and estimation accuracy. Method: We propose a novel SBI framework that integrates score matching with Langevin dynamics. To enable efficient and accurate simulation-based inference, we design a specialized score estimator featuring localized sampling to reduce simulator calls and architectural regularization that explicitly encodes statistical structure of the log-likelihood score. This estimator is coupled with gradient-based Langevin sampling for stable and efficient posterior exploration. Results: Theoretical analysis and extensive experiments demonstrate that our method significantly outperforms state-of-the-art SBI approaches on benchmark tasks and challenging posteriors—including multimodal and highly correlated distributions—achieving higher accuracy with lower simulation cost in complex posterior landscapes.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Probabilistic Inference

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Simulation-based inference (SBI) enables Bayesian analysis when the likelihood is intractable but model simulations are available. Recent advances in statistics and machine learning, including Approximate Bayesian Computation and deep generative models, have expanded the applicability of SBI, yet these methods often face challenges in moderate to high-dimensional parameter spaces. Motivated by the success of gradient-based Monte Carlo methods in Bayesian sampling, we propose a novel SBI method that integrates score matching with Langevin dynamics to explore complex posterior landscapes more efficiently in such settings. Our approach introduces tailored score-matching procedures for SBI, including a localization scheme that reduces simulation costs and an architectural regularization that embeds the statistical structure of log-likelihood scores to improve score-matching accuracy. We provide theoretical analysis of the method and illustrate its practical benefits on benchmark tasks and on more challenging problems in moderate to high dimensions, where it performs favorably compared to existing approaches.

Problem

• Efficient Bayesian inference with intractable likelihoods
• Handling moderate to high-dimensional parameter spaces
• Improving score-matching accuracy via architectural regularization

Innovation

• Uses Langevin dynamics with score matching
• Localization scheme reduces simulation costs
• Architectural regularization improves score accuracy