EmoZone-Talker: Regional Semantic Control of Audio-Driven 3DGS Talking Heads via Facial Action Units

2026-06-14

AI Summary

Existing audio-driven 3D Gaussian Splatting (3DGS) talking avatars struggle to achieve fine-grained and editable facial expression control due to modality conflicts between speech and explicit expression signals. This work formulates the challenge as a structured spatiotemporal coordination task under cross-modal conflict and introduces an anatomically informed Synergistic-Zone Prior with Attention Bias (SZ-PAB) to enable spatial semantic disentanglement. Furthermore, a Channel-Independent Temporal Action Unit Encoder (CIT-AE) is designed to model coherent expression dynamics. The proposed approach achieves, for the first time within the 3DGS framework, region-level editable expression control, significantly enhancing the accuracy, temporal consistency, and realism of upper-face expressions while preserving high-quality rendering and precise lip synchronization.

Abstract

3D Gaussian Splatting (3DGS) has shown strong potential for high-fidelity talking head synthesis. However, enabling fine-grained, interpretable, and editable facial expression control remains fundamentally challenging due to intrinsic conflicts between speech-driven facial dynamics and explicit expression signals. Existing methods rely on implicit multimodal fusion, leading to spatial entanglement and temporal instability. We present EmoZone-Talker, a novel framework that reformulates audio-driven facial animation as a structured spatial-temporal coordination problem under cross-modal conflicts. Our approach introduces an explicit spatial disentanglement and temporal dynamics modeling of facial motion. Specifically, we propose Synergy Zones with Prioritized Attention Bias (SZ-PAB) to explicitly decouple modality contributions via region-wise constraints guided by anatomical priors, and a Channel-Independent Temporal AU Encoder (CIT-AE) to model temporally coherent AU dynamics. By integrating these representations into 3D Gaussian deformation, EmoZone-Talker enables precise and interpretable control over facial expressions. Extensive experiments demonstrate that our method improves expression controllability and realism, with notable gains in upper-face accuracy and temporal coherence, while preserving high rendering quality and accurate lip synchronization. Code will be publicly released to facilitate reproducibility and further research.

Problem

• facial expression control
• audio-driven animation
• 3D Gaussian Splatting
• facial action units
• cross-modal conflict

Innovation

• 3D Gaussian Splatting
• Facial Action Units
• Spatial Disentanglement
• Temporal Dynamics
• Audio-Driven Animation