Phantom: A Unified Face-Swap Deepfake Protection Framework with Latent and Spatial Constraints

2026-06-30

AI Summary

This work addresses the limited efficacy of existing defenses against deepfake-based face swapping, which often suffer from ambiguous optimization objectives and a lack of spatial constraints, thereby inadequately protecting personal privacy. To overcome these limitations, we propose Phantom, a novel framework that synergistically integrates identity-aware optimization in the latent space with semantic mask-guided perturbation in the spatial domain. Phantom adaptively generates target identities with shifted identity features while preserving attribute consistency, confining perturbations exclusively to semantically relevant facial regions. This design effectively mitigates the suppression of adversarial signals caused by identity-style disentanglement. Extensive evaluations demonstrate substantial improvements: protection success rates increase by 27.8%, 25.6%, and 16.6% against UniFace, INSwapper, and SimSwap, respectively, in evasion scenarios, and by up to 10.2% in impersonation settings, all while significantly enhancing visual naturalness.

Abstract

Face-swapping deepfakes pose an escalating threat to personal privacy by enabling unauthorized identity manipulation. While adversarial approaches have demonstrated success against black-box face recognition (FR) models, their applicability to face-swapping scenarios remains underexplored. In particular, reliance on fixed or random targets yields ambiguous latent guidance, and the lack of explicit spatial constraints causes perturbations to spill into identity-irrelevant regions. These issues are further exacerbated by identity-style disentanglement, which suppresses adversarial signals during deepfake generation. In this paper, we present Phantom, a unified face-swap deepfake protection framework that jointly constrains perturbations in latent and spatial domains. Phantom adaptively synthesizes identity-shifted yet attribute-preserving targets to guide identity-aware latent optimization, and applies masked perturbations confined to semantically relevant facial regions. Extensive experiments on state-of-the-art face-swapping deepfakes demonstrate that Phantom improves protection success rates in dodging scenarios by 27.8%, 25.6%, and 16.6% on UniFace, INSwapper, and SimSwap, respectively, while also enhancing visual quality. Furthermore, Phantom generalizes to impersonation scenario, yielding up to 10.2% higher protection while improving perceptual fidelity. These results underscore the effectiveness of jointly leveraging latent and spatial constraints for robust and coherent facial privacy protection.

Problem

• face-swapping deepfakes
• adversarial protection
• latent constraints
• spatial constraints
• identity manipulation

Innovation

• face-swap deepfake protection
• latent-space constraint
• spatial masking
• identity-aware optimization
• adversarial perturbation