Survivability of Backdoor Attacks on Unconstrained Face Recognition Systems

2025-07-02

AI Summary

This work presents the first systematic study of backdoor attack survivability against deep learning–based face recognition systems in unconstrained, real-world settings (e.g., in-the-wild images). Addressing the end-to-end pipeline—face detection → landmark localization → feature extraction—we propose two novel detection-layer backdoor attacks: controllable face generation and landmark shifting. We demonstrate that even large-margin loss–trained feature extractors remain significantly vulnerable. By jointly exploiting face synthesis, landmark perturbation, and feature-space manipulation, we evaluate 20 system configurations across 15 attack scenarios, confirming that a single backdoor can propagate across the entire pipeline and bypass the full recognition system. Our experiments identify multiple practical, realizable attack vectors and yield deployable defense strategies and actionable recommendations. This work establishes the first system-level backdoor analysis framework for biometric security.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The widespread use of deep learning face recognition raises several security concerns. Although prior works point at existing vulnerabilities, DNN backdoor attacks against real-life, unconstrained systems dealing with images captured in the wild remain a blind spot of the literature. This paper conducts the first system-level study of backdoors in deep learning-based face recognition systems. This paper yields four contributions by exploring the feasibility of DNN backdoors on these pipelines in a holistic fashion. We demonstrate for the first time two backdoor attacks on the face detection task: face generation and face landmark shift attacks. We then show that face feature extractors trained with large margin losses also fall victim to backdoor attacks. Combining our models, we then show using 20 possible pipeline configurations and 15 attack cases that a single backdoor enables an attacker to bypass the entire function of a system. Finally, we provide stakeholders with several best practices and countermeasures.

Problem

• Study backdoor attacks on unconstrained face recognition systems
• Demonstrate face generation and landmark shift backdoor attacks
• Assess backdoor impact on feature extractors with large margin losses

Innovation

• First system-level study on face recognition backdoors
• Demonstrates face generation and landmark shift attacks
• Combines models to bypass system functions