SDO: Subspace Deconflicting Operator for Multi-Adapter Composition

2026-08-13

AI Summary

This study addresses identity confusion and attribute leakage in multi-adapter compositional generation by proposing the Subspace Disambiguation Operator (SDO). Analyzing conflict mechanisms from a parameter space perspective, SDO mitigates detrimental interference in shared layers by measuring output subspace overlap and applying equivariant transformations. As a plug-and-play module requiring no retraining, it effectively preserves distinct adapter identities. Experimental results demonstrate that SDO significantly enhances identity fidelity and compositional stability in multi-character generation. Notably, its performance advantage scales with the number of adapters, offering an efficient solution for multi-concept compositional inference.

Abstract

Composing independently trained adapters within a shared diffusion backbone provides a modular approach to multi-character generation, but naive joint deployment often causes identity mixing, cross-character attribute leakage, and unstable scene composition. We study this interference from a parameter-space perspective and hypothesize that it arises partly from conflicts between overlapping dominant subspaces in shared layers. To address this issue, we propose \textbf{SDO}, a \textbf{S}ubspace \textbf{D}econflicting \textbf{O}perator for multi-adapter composition. SDO reconstructs layer-wise low-rank updates from the selected adapters, extracts compact subspace signatures, measures pairwise conflict through output-subspace overlap, and applies a permutation-equivariant transformation that suppresses harmful shared directions while retaining identity-specific characteristics. The resulting representations are mapped back to standard adapter updates and can be directly incorporated into existing diffusion inference pipelines. Experiments demonstrate that SDO consistently improves identity fidelity and compositional stability, with particularly clear gains as the number of jointly composed adapters increases.

Problem

• Multi-Adapter Composition
• Identity Mixing
• Attribute Leakage
• Diffusion Models

Innovation

• Subspace Deconflicting Operator
• Multi-Adapter Composition
• Permutation-Equivariant Transformation
• Diffusion Models
• Identity Fidelity