Diffusion Sampling Correction via Approximately 10 Parameters

2024-11-10

arXiv.org

AI Summary

Diffusion probabilistic models (DPMs) suffer from slow sampling, and existing distillation methods rely on retraining and introduce substantial parameter overhead, limiting practical deployment. To address this, we propose PCA-based Adaptive Search (PAS), a lightweight solver correction technique that operates without retraining the backbone model. PAS performs adaptive directional correction of sampling trajectories in a low-dimensional PCA coordinate space, requiring only ~10 learnable parameters. Crucially, we empirically identify that the cumulative truncation error follows an S-shaped curve across diffusion steps—a novel observation enabling an efficient, step-adaptive search strategy. On CIFAR-10, PAS reduces the FID of DDIM (NFE=10) from 15.69 to 4.37. Training completes in under one minute on a single A100 GPU, and the method supports plug-and-play integration. PAS thus achieves superior efficiency, minimal computational overhead, and strong generalization across samplers and datasets.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision
• Constraint Satisfaction and Optimization: Search

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

While powerful for generation, Diffusion Probabilistic Models (DPMs) face slow sampling challenges, for which various distillation-based methods have been proposed. However, they typically require significant additional training costs and model parameter storage, limiting their practicality. In this work, we propose PCA-based Adaptive Search (PAS), which optimizes existing solvers for DPMs with minimal additional costs. Specifically, we first employ PCA to obtain a few basis vectors to span the high-dimensional sampling space, which enables us to learn just a set of coordinates to correct the sampling direction; furthermore, based on the observation that the cumulative truncation error exhibits an ``S"-shape, we design an adaptive search strategy that further enhances the sampling efficiency and reduces the number of stored parameters to approximately 10. Extensive experiments demonstrate that PAS can significantly enhance existing fast solvers in a plug-and-play manner with negligible costs. E.g., on CIFAR10, PAS optimizes DDIM's FID from 15.69 to 4.37 (NFE=10) using only 12 parameters and sub-minute training on a single A100 GPU. Code is available at https://github.com/onefly123/PAS.

Problem

• Reduce DPMs' slow sampling with minimal parameters
• Optimize solvers using PCA-based adaptive search
• Enhance fast solvers efficiently with negligible costs

Innovation

• PCA-based Adaptive Search (PAS) optimizes DPM solvers
• Learns 10 parameters to correct sampling direction
• Enhances fast solvers with plug-and-play efficiency