When Latents Forget Pixels: Restoring Fidelity in Diffusion Transformer Super-Resolution

2026-08-10

AI Summary

Existing latent diffusion-based super-resolution methods suffer from high-frequency detail loss due to VAE compression, making it challenging to simultaneously achieve photorealistic generation and fidelity to the input. This work proposes Pixel-Guided Super-Resolution (PGSR), a framework that extracts pixel-level evidence from the low-resolution image prior to VAE encoding and injects this information twice—during latent restoration and decoding—via conditional trajectory guidance and a decoder-side pixel anchoring mechanism. The approach trains only a lightweight restoration module while freezing the pretrained DiT and VAE, and incorporates local window attention to enhance efficiency at high resolutions. Experiments demonstrate that PGSR significantly improves consistency with the low-resolution input while preserving visual realism, outperforming current generative super-resolution methods.

Abstract

Image super-resolution (SR) with large generative models has recently achieved remarkable perceptual quality, yet maintaining fidelity to the LR observation remains challenging. In particular, we observe that diffusion transformers (DiTs) built on latent representations suffer from a critical limitation: the compression bottleneck of the VAE weakens fine-grained spatial information, leading to hallucinated details that are weakly grounded in the input image. In this work, we revisit generative SR from a representation perspective and propose a pixel-grounded super-resolution (PGSR) framework that preserves LR-observed pixel evidence before VAE compression and reuses it throughout restoration. Instead of relying solely on the compressed latent condition, PGSR extracts pre-VAE pixel evidence from the upsampled LR image and reuses it at two stages. First, Condition-Side Trajectory Guidance fuses LR-derived pixel evidence with the latent LR condition to guide the latent restoration trajectory. Second, Decoder-Side Pixel Grounding injects multi-scale pixel features into the frozen VAE decoder to ground the final rendering with LR-observed cues. To efficiently adapt large pretrained DiT models, we keep the latent autoencoder and main flow-matching backbone frozen, and train only lightweight restoration modules. We further study an efficient local-window attention variant for improved high-resolution efficiency and scalability. Extensive experiments demonstrate that PGSR improves the realism--fidelity trade-off and produces more faithful, visually convincing results than existing latent generative SR approaches.

Problem

• super-resolution
• diffusion transformer
• fidelity
• latent representation
• VAE compression

Innovation

• Pixel-Grounded Super-Resolution
• Diffusion Transformer
• Latent Compression Bottleneck
• Condition-Side Trajectory Guidance
• Decoder-Side Pixel Grounding