Robust Multi-Modal Face Anti-Spoofing with Domain Adaptation: Tackling Missing Modalities, Noisy Pseudo-Labels, and Model Degradation

2025-09-27

AI Summary

To address three key challenges in cross-domain multimodal face anti-spoofing (FAS)—modality missingness, noisy pseudo-labels, and model degradation—this paper proposes MFAS-DANet, the first domain adaptation framework tailored for multimodal FAS. It introduces a cross-modal complementary feature enhancement mechanism to mitigate modality incompleteness; designs a prediction-uncertainty-guided reliable pseudo-label generation strategy to suppress noise propagation; and incorporates an adaptive loss regulation mechanism to prevent model degradation. Evaluated on multiple benchmarks, MFAS-DANet demonstrates significantly improved generalization to unseen attack types and missing modalities. It achieves state-of-the-art (SOTA) performance, validating its effectiveness and robustness under challenging cross-domain multimodal settings.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Application Domains: Misinformation & Fake News

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Recent multi-modal face anti-spoofing (FAS) methods have investigated the potential of leveraging multiple modalities to distinguish live and spoof faces. However, pre-adapted multi-modal FAS models often fail to detect unseen attacks from new target domains. Although a more realistic domain adaptation (DA) scenario has been proposed for single-modal FAS to learn specific spoof attacks during inference, DA remains unexplored in multi-modal FAS methods. In this paper, we propose a novel framework, MFAS-DANet, to address three major challenges in multi-modal FAS under the DA scenario: missing modalities, noisy pseudo labels, and model degradation. First, to tackle the issue of missing modalities, we propose extracting complementary features from other modalities to substitute missing modality features or enhance existing ones. Next, to reduce the impact of noisy pseudo labels during model adaptation, we propose deriving reliable pseudo labels by leveraging prediction uncertainty across different modalities. Finally, to prevent model degradation, we design an adaptive mechanism that decreases the loss weight during unstable adaptations and increasing it during stable ones. Extensive experiments demonstrate the effectiveness and state-of-the-art performance of our proposed MFAS-DANet.

Problem

• Addressing missing modalities through complementary feature extraction
• Reducing noisy pseudo-label impact via uncertainty-based reliability
• Preventing model degradation with adaptive loss weighting

Innovation

• Extracting complementary features to handle missing modalities
• Deriving reliable pseudo labels using prediction uncertainty
• Designing adaptive loss weight to prevent model degradation