PGP-DiffSR: Phase-Guided Progressive Pruning for Efficient Diffusion-based Image Super-Resolution

2025-12-02

AI Summary

Diffusion-based image super-resolution methods suffer from excessive computational and memory overhead due to reliance on large-scale backbones (e.g., SDXL, DiT). To address this, we propose a phase-guided progressive pruning framework—the first to incorporate phase-aware mechanisms into diffusion model compression. Our method introduces a Phase-Swap Adaptation Module that explicitly models and fuses the frequency-domain phase information of input images to guide the progressive removal of redundant transformer blocks while restoring reconstruction capability. Crucially, the structural integrity of the diffusion process is preserved throughout pruning. Experiments demonstrate that the resulting lightweight model achieves reconstruction quality comparable to its full-sized counterpart across multiple benchmarks, while reducing computational cost and memory footprint by approximately 30%–50%. These results validate the effectiveness and generalizability of phase priors for lightweight diffusion-based super-resolution.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning on the Edge & Model Compression
• Data Mining & Knowledge Management: Data Compression

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Although diffusion-based models have achieved impressive results in image super-resolution, they often rely on large-scale backbones such as Stable Diffusion XL (SDXL) and Diffusion Transformers (DiT), which lead to excessive computational and memory costs during training and inference. To address this issue, we develop a lightweight diffusion method, PGP-DiffSR, by removing redundant information from diffusion models under the guidance of the phase information of inputs for efficient image super-resolution. We first identify the intra-block redundancy within the diffusion backbone and propose a progressive pruning approach that removes redundant blocks while reserving restoration capability. We note that the phase information of the restored images produced by the pruned diffusion model is not well estimated. To solve this problem, we propose a phase-exchange adapter module that explores the phase information of the inputs to guide the pruned diffusion model for better restoration performance. We formulate the progressive pruning approach and the phase-exchange adapter module into a unified model. Extensive experiments demonstrate that our method achieves competitive restoration quality while significantly reducing computational load and memory consumption. The code is available at https://github.com/yzb1997/PGP-DiffSR.

Problem

• Reduces computational and memory costs in diffusion-based super-resolution models
• Removes redundant blocks while preserving image restoration capability
• Uses phase information to guide pruned models for better performance

Innovation

• Progressive pruning removes redundant diffusion model blocks
• Phase-exchange adapter guides pruning using input phase information
• Unified model reduces computational load while maintaining restoration quality