Harnessing Intrinsic Subject-Aware Attention for Controllable Multi-Subject Video Generation

2026-09-10

AI Summary

本文针对多主体视频生成中的保真度控制和语义漂移问题，通过利用扩散变换器内部的固有注意力机制，提出了一种双阶段内在注意力利用框架DIAL。

Abstract

Multi-subject video generation faces two key challenges: uncontrollable fidelity strength and potential semantic drift. We address these by analyzing the internal mechanisms of Diffusion Transformers (DiTs). We found that certain attention blocks naturally form an Intrinsic Spatial Grounding Map (ISGM) that precisely locates reference subjects. Building on this insight, we propose Dual-phase Intrinsic Attention Leveraging (DIAL), a framework that uses these internal signals for both training and inference. In low-noise stages, we use ISGM to guide the attention mechanism, allowing precise control over fidelity strength during inference without retraining. In high-noise stages, we use these same maps to automatically build preference pairs at no additional cost for Reinforcement Learning (RL). This RL procedure effectively anchors the model's attention to reference subjects and mitigates semantic drift. Extensive experiments show that DIAL significantly outperforms baseline models on the OpenS2V-Eval benchmark, consistently improving identity consistency and enabling controllable fidelity strength.

Problem

• Multi-subject video generation
• uncontrollable fidelity strength
• semantic drift

Innovation

• Dual-phase Intrinsic Attention Leveraging
• Intrinsic Spatial Grounding Map
• Reinforcement Learning
• Semantic Drift Mitigation
• Fidelity Strength Control