Decoupled Guidance: Disentangling Subject and Context Pathways in Text-to-Image Personalization

2026-07-01

AI Summary

Existing text-to-image personalization methods suffer from a trade-off between fidelity and editability due to the shared conditioning pathway for subject identity and scene context. This work proposes the Decoupled Guidance (DeGu) framework, which explicitly identifies and validates this conditioning entanglement problem for the first time. DeGu employs a dual-path architecture to separately process identity and contextual information and introduces a spatial mixing mechanism for semantically aware dynamic fusion. The framework is plug-and-play, requiring no modification to the backbone model, and is compatible with mainstream generative architectures such as DiT. It enables flexible adjustment of the fidelity–editability balance during inference and achieves significant performance gains across diverse personalization settings and models.

Abstract

Text-to-image personalization aims to generate a user-provided subject in novel scenes described by text. However, most existing methods encode subject identity (fidelity) and context (editability) through the same conditioning pathway, forcing the two to compete for attention-map resources. We refer to this phenomenon as conditioning entanglement and show that it induces a fidelity-editability trade-off. We further provide causal evidence by replacing the target subject token with a generic subject token, which produces shifts in attention allocation and corresponding changes in context adherence. To this end, we propose Decoupled Guidance (DeGu), a plug-and-play framework that routes subject identity and scene context through two independent guidance streams. We further introduce a spatial mixing mechanism that dynamically fuses these streams, ensuring each operates within its semantically relevant region without interference. Furthermore, DeGu can be readily applied to existing personalization methods without modifying the underlying backbone models, consistently improving the overall personalization performance while enabling inference-time control over the fidelity-editability balance, across diverse methods and backbones, including flow-matching Diffusion Transformers (DiTs).

Problem

• text-to-image personalization
• conditioning entanglement
• fidelity-editability trade-off
• subject identity
• context adherence

Innovation

• Decoupled Guidance
• conditioning entanglement
• text-to-image personalization
• spatial mixing
• fidelity-editability trade-off