FrequencyBooster: Full-Frequency Modeling for High-Fidelity Pixel Diffusion

2026-05-17

AI Summary

Existing pixel-level diffusion models struggle to balance computational efficiency with the preservation of high-frequency details, often suffering from texture distortion due to localized decoding. This work proposes FrequencyBooster, a novel framework that achieves efficient full-spectrum joint modeling in pixel-space diffusion for the first time. By synergistically optimizing a high-capacity high-frequency decoder with a Diffusion Transformer backbone, the method simultaneously maintains global semantic coherence and local detail fidelity. Breaking away from conventional spectral trade-offs, FrequencyBooster attains state-of-the-art performance with FID scores of 1.60 (at 320 sampling steps) and 1.69 on ImageNet at 256×256 and 512×512 resolutions, respectively, significantly outperforming current pixel-level and latent-space generative models.

Abstract

To circumvent the inherent fidelity bottlenecks and optimization misalignment of VAE-based latent diffusion, pixel-space diffusion models have emerged as a compelling end-to-end paradigm. However, existing pixel diffusion models often struggle to balance computational efficiency with the preservation of high-frequency details. They frequently resort to patch-based compression or restricted local decoding, leading to a "spectral compromise" where high-frequency and fine-grained pixel information are suppressed. To address these challenges, we propose \textbf{FrequencyBooster}, a novel framework designed to empower pixel diffusion with full-frequency modeling capabilities without prohibitive overhead. The core of our method is a high-capacity decoder that specializes in extracting exhaustive high-frequency details and low-frequency semantics, the latter of which is derived from a Diffusion Transformer (DiT) backbone. Unlike prior works that sacrifice global context for local refinement, FrequencyBooster leverages high-dimensional feature representations to maintain global structural integrity while achieving superior pixel-level precision. Extensive experiments on ImageNet demonstrate the effectiveness of our approach: our model achieves a state-of-the-art FID of \textbf{1.60} at $256 \times 256$ resolution within only 320 epochs. Furthermore, at $512 \times 512$ resolution, FrequencyBooster attains an FID of \textbf{1.69}, significantly outperforming existing pixel-space and latent-space generative models.

Problem

• pixel diffusion
• high-frequency details
• spectral compromise
• computational efficiency
• full-frequency modeling

Innovation

• pixel diffusion
• full-frequency modeling
• high-frequency details
• Diffusion Transformer
• generative modeling