Ensemble Kalman Diffusion Guidance: A Derivative-free Method for Inverse Problems

2024-09-30

arXiv.org

AI Summary

Solving black-box inverse problems—such as fluid inversion and astronomical imaging—where analytical derivatives or explicit forward models are unavailable remains challenging. Existing diffusion-based methods often rely on gradient computation, pseudo-inversion, or restrictive assumptions about model architecture. Method: This paper introduces a fully gradient-free, pseudo-inverse-free, and model-agnostic diffusion framework. Its core innovation is the first integration of ensemble Kalman filter principles into diffusion sampling, enabling efficient guidance of black-box forward processes via forward operator evaluations and a pre-trained diffusion prior. The method combines stochastic differential equation discretization with implicit prior modeling to achieve robust posterior sampling without requiring derivative information. Results: Experiments demonstrate that our approach significantly outperforms state-of-the-art gradient-based diffusion methods on strongly nonlinear scientific inverse problems, achieving higher reconstruction fidelity. It provides a general, robust, and plug-and-play solution paradigm for derivative-inaccessible scenarios.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

When solving inverse problems, it is increasingly popular to use pre-trained diffusion models as plug-and-play priors. This framework can accommodate different forward models without re-training while preserving the generative capability of diffusion models. Despite their success in many imaging inverse problems, most existing methods rely on privileged information such as derivative, pseudo-inverse, or full knowledge about the forward model. This reliance poses a substantial limitation that restricts their use in a wide range of problems where such information is unavailable, such as in many scientific applications. To address this issue, we propose Ensemble Kalman Diffusion Guidance (EnKG) for diffusion models, a derivative-free approach that can solve inverse problems by only accessing forward model evaluations and a pre-trained diffusion model prior. We study the empirical effectiveness of our method across various inverse problems, including scientific settings such as inferring fluid flows and astronomical objects, which are highly non-linear inverse problems that often only permit black-box access to the forward model.

Problem

• Solves inverse problems without derivative or forward model knowledge
• Uses pre-trained diffusion models as priors for diverse forward models
• Addresses non-linear scientific inverse problems with black-box access

Innovation

• Uses Ensemble Kalman for derivative-free guidance
• Leverages pre-trained diffusion models as priors
• Solves inverse problems with black-box forward models