Solving Inverse Problems with FLAIR

2025-06-03

AI Summary

Current flow-based latent variable generative models (e.g., Stable Diffusion 3) suffer from limited fidelity in inverse imaging tasks—stemming from nonlinear forward mappings, intractable data likelihoods, and difficulty recovering rare data modes. To address this, we propose FLAIR, a training-free variational framework. Its key contributions are: (1) the first degradation-agnostic flow-matching variational objective; (2) a deterministic trajectory adjustment mechanism that explicitly enhances reconstruction of atypical data patterns; and (3) decoupled optimization of data fidelity and regularization, coupled with a time-varying calibration strategy to dynamically modulate regularization strength. Evaluated on standard inverse imaging benchmarks—including deblurring, super-resolution, and inpainting—FLAIR consistently outperforms state-of-the-art diffusion- and flow-based methods, achieving significant improvements in both reconstruction quality (e.g., PSNR, SSIM) and sample diversity (e.g., LPIPS, FID).

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Flow-based latent generative models such as Stable Diffusion 3 are able to generate images with remarkable quality, even enabling photorealistic text-to-image generation. Their impressive performance suggests that these models should also constitute powerful priors for inverse imaging problems, but that approach has not yet led to comparable fidelity. There are several key obstacles: (i) the encoding into a lower-dimensional latent space makes the underlying (forward) mapping non-linear; (ii) the data likelihood term is usually intractable; and (iii) learned generative models struggle to recover rare, atypical data modes during inference. We present FLAIR, a novel training free variational framework that leverages flow-based generative models as a prior for inverse problems. To that end, we introduce a variational objective for flow matching that is agnostic to the type of degradation, and combine it with deterministic trajectory adjustments to recover atypical modes. To enforce exact consistency with the observed data, we decouple the optimization of the data fidelity and regularization terms. Moreover, we introduce a time-dependent calibration scheme in which the strength of the regularization is modulated according to off-line accuracy estimates. Results on standard imaging benchmarks demonstrate that FLAIR consistently outperforms existing diffusion- and flow-based methods in terms of reconstruction quality and sample diversity.

Problem

• Overcoming non-linear mapping in flow-based inverse imaging
• Addressing intractable data likelihood in generative models
• Recovering rare data modes in inference with FLAIR

Innovation

• Training-free variational framework for inverse problems
• Variational objective agnostic to degradation type
• Time-dependent calibration for regularization strength