Progressive Multi-Source Domain Adaptation for Personalized Facial Expression Recognition

2025-04-05

AI Summary

Personalized facial expression recognition (FER) faces significant challenges in unsupervised multi-source-to-single-target domain adaptation, including large inter-domain distribution shifts, misalignment between source domains and the target subject, and severe negative transfer. To address these issues, this paper proposes a progressive multi-source domain adaptation framework: (1) dynamically ranking and sequentially incorporating source knowledge based on representation similarity between each source domain and the target subject; (2) introducing a density-aware memory mechanism that selectively replays samples from high-density regions in feature space to mitigate catastrophic forgetting; and (3) jointly optimizing multi-source unsupervised domain adaptation (MSDA) with feature-level distribution alignment. Evaluated on the BioVid and UNBC-McMaster pain datasets, the method achieves substantial improvements in recognition accuracy, effectively suppresses negative transfer, reduces computational redundancy, and demonstrates strong generalizability and practical applicability.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Multi-modal Vision
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Personalized facial expression recognition (FER) involves adapting a machine learning model using samples from labeled sources and unlabeled target domains. Given the challenges of recognizing subtle expressions with considerable interpersonal variability, state-of-the-art unsupervised domain adaptation (UDA) methods focus on the multi-source UDA (MSDA) setting, where each domain corresponds to a specific subject, and improve model accuracy and robustness. However, when adapting to a specific target, the diverse nature of multiple source domains translates to a large shift between source and target data. State-of-the-art MSDA methods for FER address this domain shift by considering all the sources to adapt to the target representations. Nevertheless, adapting to a target subject presents significant challenges due to large distributional differences between source and target domains, often resulting in negative transfer. In addition, integrating all sources simultaneously increases computational costs and causes misalignment with the target. To address these issues, we propose a progressive MSDA approach that gradually introduces information from subjects based on their similarity to the target subject. This will ensure that only the most relevant sources from the target are selected, which helps avoid the negative transfer caused by dissimilar sources. We first exploit the closest sources to reduce the distribution shift with the target and then move towards the furthest while only considering the most relevant sources based on the predetermined threshold. Furthermore, to mitigate catastrophic forgetting caused by the incremental introduction of source subjects, we implemented a density-based memory mechanism that preserves the most relevant historical source samples for adaptation. Our experiments show the effectiveness of our proposed method on pain datasets: Biovid and UNBC-McMaster.

Problem

• Adapting FER models to personalized target domains
• Reducing domain shift between multiple sources and target
• Avoiding negative transfer from dissimilar source domains

Innovation

• Progressive MSDA for gradual source integration
• Density-based memory to prevent catastrophic forgetting
• Threshold-based selection of relevant sources