A Quality-Guided Mixture of Score-Fusion Experts Framework for Human Recognition

2025-07-31

AI Summary

In multimodal whole-body biometrics, conventional score-level fusion methods—such as weighted averaging—fail to account for inter-modal score distribution discrepancies and input data quality variations. To address this, we propose Quality-guided Mixture-of-Experts (QME), a novel score fusion framework that integrates modality-specific quality estimators (QEs) and a pseudo-quality loss, jointly optimized with a triplet loss over fused scores, within a Mixture-of-Experts (MoE) architecture. QME enables dynamic, quality-aware, and adaptive score fusion, significantly enhancing fusion robustness and metric discriminability while mitigating model misalignment and degradation caused by low-quality inputs. Extensive experiments on multiple whole-body biometric benchmark datasets demonstrate that QME consistently outperforms state-of-the-art methods, achieving new SOTA performance. These results validate both the effectiveness and generalizability of quality-aware fusion as a principled paradigm for multimodal biometric recognition.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Mixture of Experts (MoE)

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Whole-body biometric recognition is a challenging multimodal task that integrates various biometric modalities, including face, gait, and body. This integration is essential for overcoming the limitations of unimodal systems. Traditionally, whole-body recognition involves deploying different models to process multiple modalities, achieving the final outcome by score-fusion (e.g., weighted averaging of similarity matrices from each model). However, these conventional methods may overlook the variations in score distributions of individual modalities, making it challenging to improve final performance. In this work, we present extbf{Q}uality-guided extbf{M}ixture of score-fusion extbf{E}xperts (QME), a novel framework designed for improving whole-body biometric recognition performance through a learnable score-fusion strategy using a Mixture of Experts (MoE). We introduce a novel pseudo-quality loss for quality estimation with a modality-specific Quality Estimator (QE), and a score triplet loss to improve the metric performance. Extensive experiments on multiple whole-body biometric datasets demonstrate the effectiveness of our proposed approach, achieving state-of-the-art results across various metrics compared to baseline methods. Our method is effective for multimodal and multi-model, addressing key challenges such as model misalignment in the similarity score domain and variability in data quality.

Problem

• Improving whole-body biometric recognition via learnable score-fusion
• Addressing score distribution variations across biometric modalities
• Overcoming model misalignment and data quality variability

Innovation

• Quality-guided Mixture of Experts framework
• Modality-specific Quality Estimator
• Score triplet loss for metric improvement