GaussianSwap: Animatable Video Face Swapping with 3D Gaussian Splatting

2026-01-09

arXiv.org

AI Summary

This work proposes a novel approach to video face swapping by introducing 3D Gaussian splatting to generate animatable and interactively controllable facial avatars. Unlike existing methods that produce static, pixel-level outputs, the proposed framework estimates FLAME parameters and camera poses from the target video, binds 3D Gaussians to the FLAME model, and integrates identity features from the source image to achieve high-fidelity, identity-preserving reconstructions. By moving beyond conventional pixel-based generation, the method excels in identity consistency, visual sharpness, and temporal stability. Furthermore, it enables real-time interaction and pose-driven animation, offering a significant advancement toward dynamic and controllable face reenactment.

Technology Category

• Computer Vision: 3D Computer Vision
• Humans and AI: Interaction Techniques and Devices
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We introduce GaussianSwap, a novel video face swapping framework that constructs a 3D Gaussian Splatting based face avatar from a target video while transferring identity from a source image to the avatar. Conventional video swapping frameworks are limited to generating facial representations in pixel-based formats. The resulting swapped faces exist merely as a set of unstructured pixels without any capacity for animation or interactive manipulation. Our work introduces a paradigm shift from conventional pixel-based video generation to the creation of high-fidelity avatar with swapped faces. The framework first preprocesses target video to extract FLAME parameters, camera poses and segmentation masks, and then rigs 3D Gaussian splats to the FLAME model across frames, enabling dynamic facial control. To ensure identity preserving, we propose an compound identity embedding constructed from three state-of-the-art face recognition models for avatar finetuning. Finally, we render the face-swapped avatar on the background frames to obtain the face-swapped video. Experimental results demonstrate that GaussianSwap achieves superior identity preservation, visual clarity and temporal consistency, while enabling previously unattainable interactive applications.

Problem

• video face swapping
• animatable avatar
• pixel-based representation
• interactive manipulation
• identity preservation

Innovation

• 3D Gaussian Splatting
• animatable face avatar
• video face swapping
• identity preservation
• FLAME model