DCR: Counterfactual Attractor Guidance for Rare Compositional Generation

2026-05-07

AI Summary

Diffusion models often degrade into generating common configurations when tasked with producing rare yet plausible semantic combinations—such as snowy beaches or nocturnal rainbows—due to inherent default completion biases. This work proposes DCR, a training-free framework that explicitly models and suppresses such biases during the diffusion process. By constructing counterfactual attractors to characterize the model’s default tendencies, DCR dynamically corrects denoising trajectories in latent space through trajectory discrepancy analysis and a projection-based repulsion mechanism, effectively steering generation away from high-frequency semantic drifts that conflict with the target concept. Operating solely within the standard sampling pipeline without requiring model retraining or architectural modifications, DCR significantly enhances the fidelity of rare compositions while preserving overall generation quality, thereby enabling controllable intervention on the model’s intrinsic biases.

Abstract

Diffusion models generate realistic visual content, yet often fail to produce rare but plausible compositions. When prompted with combinations that are valid but underrepresented in training data, such as a snowy beach or a rainbow at night, the generation process frequently collapses toward more common alternatives. We identify this failure mode as default completion bias, where denoising trajectories are implicitly attracted toward high-frequency semantic configurations. Existing guidance mechanisms do not explicitly model this competing tendency and therefore struggle to prevent such collapse. We introduce Default Completion Repulsion (DCR), a training-free framework that explicitly models and suppresses default completion behavior. DCR constructs a counterfactual attractor by relaxing the rare compositional factor while preserving surrounding semantics, inducing an alternative denoising trajectory reflecting the model's preferred completion. We define the discrepancy between target and attractor trajectories as a counterfactual drift, and propose a projection-based repulsion mechanism that removes guidance components aligned with this drift direction. This suppresses undesired frequent completions while preserving other semantic components. DCR operates entirely within the standard diffusion sampling process without retraining or architectural modification. Experiments on rare compositional prompts show that DCR improves compositional fidelity while maintaining visual quality. Our analysis further shows that the framework exposes and counteracts intrinsic model biases, offering a new perspective on controllable generation beyond explicit constraint enforcement.

Problem

• rare compositional generation
• default completion bias
• diffusion models
• counterfactual attractor
• compositional fidelity

Innovation

• Default Completion Repulsion
• counterfactual attractor
• diffusion models
• compositional generation
• guidance mechanism