Adaptive Moments are Surprisingly Effective for Plug-and-Play Diffusion Sampling

2026-03-17

AI Summary

This work addresses the significant noise introduced by approximating likelihood gradients in guided diffusion sampling, which undermines sampling stability and generation quality. The paper proposes the first concise and effective integration of adaptive moment estimation—inspired by Adam-like optimizers—into the guidance phase of Plug-and-Play diffusion sampling to dynamically denoise and stabilize noisy likelihood gradients. Requiring no computationally expensive operations, the method substantially outperforms existing costly approaches and achieves state-of-the-art performance on both image restoration and class-conditional generation tasks. By enhancing gradient alignment and sample fidelity, the proposed technique effectively improves the overall quality and reliability of generated samples.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Guided diffusion sampling relies on approximating often intractable likelihood scores, which introduces significant noise into the sampling dynamics. We propose using adaptive moment estimation to stabilize these noisy likelihood scores during sampling. Despite its simplicity, our approach achieves state-of-the-art results on image restoration and class-conditional generation tasks, outperforming more complicated methods, which are often computationally more expensive. We provide empirical analysis of our method on both synthetic and real data, demonstrating that mitigating gradient noise through adaptive moments offers an effective way to improve alignment.

Problem

• guided diffusion sampling
• likelihood scores
• gradient noise
• sampling dynamics
• image restoration

Innovation

• adaptive moments
• diffusion sampling
• gradient noise
• plug-and-play
• likelihood scores