Generalized Single-Image-Based Morphing Attack Detection Using Deep Representations from Vision Transformer

2024-06-17

2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

AI Summary

Detecting single-image face forgery in open-set scenarios—where unknown generative algorithms, post-processing techniques, and acquisition devices severely degrade generalization—remains challenging. Method: This paper introduces Vision Transformers (ViTs) to this task for the first time, leveraging their global-local joint modeling capability to precisely capture sparse, fine-grained fusion artifacts across the entire face. We propose an end-to-end supervised training framework and adopt a rigorous cross-dataset evaluation protocol to assess generalization. Results: Our method significantly outperforms CNN-based baselines under cross-dataset testing and achieves state-of-the-art performance on intra-dataset benchmarks. This work establishes a robust, scalable liveness detection paradigm for high-security applications such as border control and identity document verification.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Face morphing attacks have posed severe threats to Face Recognition Systems (FRS), which are operated in border control and passport issuance use cases. Correspondingly, morphing attack detection algorithms (MAD) are needed to defend against such attacks. MAD approaches must be robust enough to handle unknown attacks in an open-set scenario where attacks can originate from various morphing generation algorithms, post-processing and the diversity of printers/scanners. The problem of generalization is further pronounced when the detection has to be made on a single suspected image. In this paper, we propose a generalized single-image-based MAD (S-MAD) algorithm by learning the encoding from Vision Transformer (ViT) architecture. Compared to CNN-based architectures, ViT model has the advantage on integrating local and global information and hence can be suitable to detect the morphing traces widely distributed among the face region. Extensive experiments are carried out on face morphing datasets generated using publicly available FRGC face datasets. Several state-of-the-art (SOTA) MAD algorithms, including representative ones that have been publicly evaluated, have been selected and benchmarked with our ViT-based approach. Obtained results demonstrate the improved detection performance of the proposed S-MAD method on inter-dataset testing (when different data is used for training and testing) and comparable performance on intra-dataset testing (when the same data is used for training and testing) experimental protocol.

Problem

• Facial Deformation Attack Detection
• Enhanced Facial Recognition Security
• Single Image Processing

Innovation

• Single-Image Deformation Attack Detection
• Visual Transformer
• Superior Performance in MAD