SAIP: A Plug-and-Play Scale-adaptive Module in Diffusion-based Inverse Problems

2025-09-29

AI Summary

Existing diffusion-based inverse problem solvers rely on manually set, fixed-scale weighting to balance prior and likelihood terms, limiting adaptability across timesteps and diverse tasks—thus hindering generalization and reconstruction fidelity. To address this, we propose a plug-and-play temporal adaptive scaling module that, for the first time, introduces a training-free dynamic weighting mechanism grounded in Bayesian inference. This module automatically optimizes the trade-off between prior and likelihood scores by fusing them in real time during sampling. It is fully compatible with mainstream samplers—including DPS, DMPS, and πGDM—without modifying the underlying diffusion model architecture. Extensive experiments on diverse image restoration tasks, particularly under complex degradations (e.g., blind super-resolution, non-uniform motion blur), demonstrate substantial improvements in reconstruction quality. The method achieves strong generalizability, seamless integration, and computational efficiency—requiring no additional training or architectural changes.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Solving inverse problems with diffusion models has shown promise in tasks such as image restoration. A common approach is to formulate the problem in a Bayesian framework and sample from the posterior by combining the prior score with the likelihood score. Since the likelihood term is often intractable, estimators like DPS, DMPS, and $π$GDM are widely adopted. However, these methods rely on a fixed, manually tuned scale to balance prior and likelihood contributions. Such a static design is suboptimal, as the ideal balance varies across timesteps and tasks, limiting performance and generalization. To address this issue, we propose SAIP, a plug-and-play module that adaptively refines the scale at each timestep without retraining or altering the diffusion backbone. SAIP integrates seamlessly into existing samplers and consistently improves reconstruction quality across diverse image restoration tasks, including challenging scenarios.

Problem

• Adaptively balances prior and likelihood in diffusion-based inverse problems
• Eliminates manual scale tuning limitations across timesteps and tasks
• Enhances image restoration quality without retraining diffusion models

Innovation

• Adaptive scale module for diffusion inverse problems
• Plug-and-play integration without retraining backbone
• Timestep-wise scale refinement for image restoration