Uncertainty-Aware Label Refinement on Hypergraphs for Personalized Federated Facial Expression Recognition

2025-01-03

IEEE transactions on circuits and systems for video technology (Print)

AI Summary

In privacy-sensitive federated facial expression recognition (FER), performance is hindered by data silos, label noise, and client heterogeneity. To address these challenges, this paper proposes a hypergraph-based uncertainty-aware label refinement framework. Methodologically, it introduces hypergraph neural networks—first applied to federated FER—to explicitly model high-order semantic relationships among samples; further, it designs a personalized uncertainty estimation module coupled with an end-to-end trainable local label propagation mechanism, enabling client-adaptive label optimization. Experiments on two real-world FER benchmarks demonstrate that the proposed method significantly outperforms existing state-of-the-art approaches. Results validate that hypergraph modeling effectively captures client heterogeneity-induced uncertainty and corrects noisy labels, thereby improving robustness and generalization in federated FER.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Bias, Fairness & Privacy
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Most facial expression recognition (FER) models are trained on large-scale expression data with centralized learning. Unfortunately, collecting a large amount of centralized expression data is difficult in practice due to privacy concerns of facial images. In this paper, we investigate FER under the framework of personalized federated learning, which is a valuable and practical decentralized setting for real-world applications. To this end, we develop a novel uncertainty-Aware label refineMent on hYpergraphs (AMY) method. For local training, each local model consists of a backbone, an uncertainty estimation (UE) block, and an expression classification (EC) block. In the UE block, we leverage a hypergraph to model complex high-order relationships between expression samples and incorporate these relationships into uncertainty features. A personalized uncertainty estimator is then introduced to estimate reliable uncertainty weights of samples in the local client. In the EC block, we perform label propagation on the hypergraph, obtaining high-quality refined labels for retraining an expression classifier. Based on the above, we effectively alleviate heterogeneous sample uncertainty across clients and learn a robust personalized FER model in each client. Experimental results on two challenging real-world facial expression databases show that our proposed method consistently outperforms several state-of-the-art methods. This indicates the superiority of hypergraph modeling for uncertainty estimation and label refinement on the personalized federated FER task. The source code will be released at https://github.com/mobei1006/AMY.

Problem

• Facial Expression Recognition
• Personalized Federated Learning
• Data Privacy

Innovation

• AMY
• Uncertainty-aware Label Refinement
• Personalized Federated Learning