Diffusion-Driven Deceptive Patches: Adversarial Manipulation and Forensic Detection in Facial Identity Verification

2026-01-14

AI Summary

This work addresses the vulnerability of facial biometric systems to adversarial attacks by proposing a highly realistic and interpretable adversarial patch generation method. By integrating diffusion models with the Fast Gradient Sign Method (FGSM), the approach simultaneously optimizes adversarial perturbations while performing luminance correction and Gaussian smoothing, achieving high visual fidelity (SSIM of 0.95). The framework innovatively incorporates a ViT-GPT2 architecture to translate identity information into semantic descriptions, thereby enhancing forensic interpretability. Furthermore, it systematically evaluates the robustness of recognition models under adversarial conditions through a fusion of perceptual hashing and image segmentation techniques. This end-to-end pipeline balances high attack efficacy with strong interpretability, offering a novel direction for improving the security of biometric authentication systems.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Large-scale security measurements

Abstract

This work presents an end-to-end pipeline for generating, refining, and evaluating adversarial patches to compromise facial biometric systems, with applications in forensic analysis and security testing. We utilize FGSM to generate adversarial noise targeting an identity classifier and employ a diffusion model with reverse diffusion to enhance imperceptibility through Gaussian smoothing and adaptive brightness correction, thereby facilitating synthetic adversarial patch evasion. The refined patch is applied to facial images to test its ability to evade recognition systems while maintaining natural visual characteristics. A Vision Transformer (ViT)-GPT2 model generates captions to provide a semantic description of a person's identity for adversarial images, supporting forensic interpretation and documentation for identity evasion and recognition attacks. The pipeline evaluates changes in identity classification, captioning results, and vulnerabilities in facial identity verification and expression recognition under adversarial conditions. We further demonstrate effective detection and analysis of adversarial patches and adversarial samples using perceptual hashing and segmentation, achieving an SSIM of 0.95.

Problem

• adversarial patches
• facial identity verification
• forensic detection
• identity evasion
• biometric security

Innovation

• diffusion model
• adversarial patch
• Vision Transformer
• perceptual hashing
• facial identity verification