On-the-fly Repulsion in the Contextual Space for Rich Diversity in Diffusion Transformers

2026-03-30

AI Summary

While existing text-to-image diffusion models achieve strong semantic alignment, their generation diversity is constrained by typicality bias. This work proposes a lightweight contextual spatial repulsion mechanism integrated into the multimodal attention modules of intermediate layers in Diffusion Transformers. The intervention occurs at a critical stage—after structural information has been incorporated but before compositional features become fixed—thereby steering the generation trajectory without altering inputs or requiring additional optimization. To our knowledge, this is the first method to enable real-time repulsion during forward propagation. It substantially enhances output diversity while preserving high visual quality, semantic fidelity, and computational efficiency, with consistent effectiveness even in Turbo and distilled model variants.

Abstract

Modern Text-to-Image (T2I) diffusion models have achieved remarkable semantic alignment, yet they often suffer from a significant lack of variety, converging on a narrow set of visual solutions for any given prompt. This typicality bias presents a challenge for creative applications that require a wide range of generative outcomes. We identify a fundamental trade-off in current approaches to diversity: modifying model inputs requires costly optimization to incorporate feedback from the generative path. In contrast, acting on spatially-committed intermediate latents tends to disrupt the forming visual structure, leading to artifacts. In this work, we propose to apply repulsion in the Contextual Space as a novel framework for achieving rich diversity in Diffusion Transformers. By intervening in the multimodal attention channels, we apply on-the-fly repulsion during the transformer's forward pass, injecting the intervention between blocks where text conditioning is enriched with emergent image structure. This allows for redirecting the guidance trajectory after it is structurally informed but before the composition is fixed. Our results demonstrate that repulsion in the Contextual Space produces significantly richer diversity without sacrificing visual fidelity or semantic adherence. Furthermore, our method is uniquely efficient, imposing a small computational overhead while remaining effective even in modern "Turbo" and distilled models where traditional trajectory-based interventions typically fail.

Problem

• diversity
• text-to-image generation
• diffusion models
• typicality bias
• generative variety

Innovation

• Contextual Space
• on-the-fly repulsion
• Diffusion Transformers
• generative diversity
• multimodal attention