SyncCache: Exploiting Asymmetric Dynamics for Fast Audio-Driven Portrait Animation

2026-06-29

AI Summary

This work addresses the high inference latency of Diffusion Transformers (DiT) in audio-driven portrait animation, a challenge exacerbated by existing training-free caching methods that overlook the asymmetric spatiotemporal dynamics across spatial regions and modalities. To overcome this limitation, we propose SyncCache, a training-free acceleration framework tailored for DiT-based systems. SyncCache employs spatially asymmetric probing to identify facial regions exhibiting high-frequency audio-driven dynamics and integrates a modality-decoupled caching mechanism that prioritizes processing of critical content while caching stable background regions. The cache selection strategy is formulated as an offline dynamic programming problem, enhanced with residual reuse and lightweight audio-block recomputation. Evaluated on HunyuanVideo-Avatar and Wan-S2V benchmarks, SyncCache achieves speedups of up to 4.12× and 3.75×, respectively, with negligible visual quality degradation and preserved lip-sync accuracy.

Abstract

Diffusion Transformers (DiTs) have significantly advanced audio-driven portrait animation, but their high computational cost leads to substantial inference latency. Although training-free diffusion caching accelerates inference significant, existing methods are primarily developed for text-conditioned generation and overlook the spatial and modality imbalances inherent in audio-driven portrait animation. In this paper, we propose SyncCache, a training-free caching acceleration method tailored for DiT-based portrait animation that explicitly exploits asymmetric dynamics. Specifically, high-frequency dynamics driven by audio conditions and concentrated in human regions are more challenging and critical to cache and reuse than the low-frequency visual background in portrait animation. First, we introduce Spatially-Asymmetric Probing to prioritize error sensitivity in dynamic human region. Second, through Modality-Decoupled Caching, we bypass heavy DiT block by reusing stable inter-block residuals, while continuously recomputing lightweight audio blocks to preserve precise lip synchronization. Furthermore, we introduce a cache ratio to control cache capacity and formulate memory-adaptive cache selection as an offline dynamic programming problem without online overhead. Extensive experiments demonstrate that SyncCache achieves superior speed-quality trade-offs, delivering up to 4.12x acceleration on HunyuanVideo-Avatar and 3.75x on Wan-S2V with near-lossless visual fidelity and precise audio alignment.

Problem

• audio-driven portrait animation
• diffusion transformers
• inference acceleration
• asymmetric dynamics
• caching

Innovation

• SyncCache
• Diffusion Transformer
• audio-driven animation
• asymmetric dynamics
• training-free caching