ResDiT: Evoking the Intrinsic Resolution Scalability in Diffusion Transformers

2025-12-01

AI Summary

Pretrained diffusion Transformers (DiTs) suffer from spatial layout collapse and texture distortion when generating high-resolution (HR) images. This paper proposes ResDiT, a training-free method that enables efficient resolution scaling by analyzing the intrinsic generation mechanism of DiTs. Our key contributions are: (1) identifying positional encoding (PE) as the primary cause of layout degradation and introducing a PE scaling strategy to recalibrate spatial awareness at HR; and (2) designing a local attention enhancement module that integrates block-wise feature fusion with Gaussian-weighted stitching to eliminate grid artifacts while preserving fine-grained local details. ResDiT significantly improves HR generation quality across text-to-image synthesis and image super-resolution tasks, maintains strong spatial controllability, outperforms existing zero-shot upscaling approaches, and operates in a plug-and-play manner without fine-tuning.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Leveraging pre-trained Diffusion Transformers (DiTs) for high-resolution (HR) image synthesis often leads to spatial layout collapse and degraded texture fidelity. Prior work mitigates these issues with complex pipelines that first perform a base-resolution (i.e., training-resolution) denoising process to guide HR generation. We instead explore the intrinsic generative mechanisms of DiTs and propose ResDiT, a training-free method that scales resolution efficiently. We identify the core factor governing spatial layout, position embeddings (PEs), and show that the original PEs encode incorrect positional information when extrapolated to HR, which triggers layout collapse. To address this, we introduce a PE scaling technique that rectifies positional encoding under resolution changes. To further remedy low-fidelity details, we develop a local-enhancement mechanism grounded in base-resolution local attention. We design a patch-level fusion module that aggregates global and local cues, together with a Gaussian-weighted splicing strategy that eliminates grid artifacts. Comprehensive evaluations demonstrate that ResDiT consistently delivers high-fidelity, high-resolution image synthesis and integrates seamlessly with downstream tasks, including spatially controlled generation.

Problem

• ResDiT addresses layout collapse in high-resolution image synthesis.
• It corrects positional encoding errors during resolution scaling.
• The method enhances texture fidelity without requiring training.

Innovation

• PE scaling technique rectifies positional encoding
• Local-enhancement mechanism uses base-resolution attention
• Patch-level fusion aggregates global and local cues