Controlling Motion Transfer in Diffusion Transformers via Attention Heads

2026-07-13

AI Summary

This work addresses the challenge of aligning motion transfer with target semantics and spatial structure in video diffusion transformers by proposing a training-free, controllable motion transfer method. By analyzing the functional specialization of attention heads, the approach decouples motion and spatial structure at the attention layer for the first time, identifying dedicated motion and structure heads. Leveraging semantic correspondence guidance and selective feature injection, it enables precise control over motion transfer without requiring any parameter updates. The method offers strong interpretability and generates high-fidelity videos that maintain temporal coherence, accurately reproduce reference motion, and preserve the structural consistency of the target content.

Abstract

Diffusion Transformers (DiTs) have advanced video generation with high-quality, temporally coherent results. However, extending them to motion transfer, which requires following reference motion while aligning with a target prompt, remains challenging due to limited understanding of motion and structure representations within DiTs. We analyze video DiTs at the attention-head level and identify distinct heads specialized for motion and spatial structure. Based on this insight, we propose a head-aware controllable motion transfer framework that requires no parameter updates. Our method refines motion cues from motion-specialized heads via semantic correspondence guidance and preserves structure through selective feature injection. This head-level control not only enables accurate motion transfer but also provides an interpretable foundation for controllable video generation with DiTs.

Problem

• motion transfer
• Diffusion Transformers
• video generation
• attention heads
• controllable generation

Innovation

• Diffusion Transformers
• motion transfer
• attention heads
• controllable generation
• semantic correspondence