Towards Robust Uncertainty-Aware Speaker Modeling

2026-07-06

AI Summary

This work addresses the challenge of unreliable uncertainty estimation and poor calibration in speaker modeling under domain shift. To enhance the reliability of speaker embeddings, the authors propose a robust uncertainty modeling framework that improves both uncertainty estimation and adaptation. The core innovations include an Inter- and Intra-Speaker-Aware Uncertainty Softmax mechanism that jointly captures speaker separability and intra-speaker variability, along with an Uncertainty-Calibrated Domain Adaptation (UCDA) framework. Experimental results demonstrate that the proposed approach significantly improves the reliability of uncertainty estimates and enhances speaker recognition robustness on both in-domain and cross-domain benchmarks.

Abstract

Speaker embeddings aggregate frame-level acoustic features into compact representations for speaker recognition. Recent uncertainty-aware speaker modeling approaches further characterize the reliability of speaker embeddings by estimating their associated uncertainty. However, existing methods often suffer from inaccurate uncertainty estimation and uncertainty miscalibration under domain shifts. To address these challenges, we propose a robust uncertainty modeling framework from both estimation and adaptation perspectives. Specifically, we introduce an Inter- and Intra-Speaker-Aware Uncertainty Softmax that incorporates both inter-speaker separability and intra-speaker variability into uncertainty learning, enabling uncertainty estimates to better capture the reliability of speaker embeddings. Furthermore, we propose an Uncertainty-Calibrated Domain Adaptation (UCDA) framework to mitigate uncertainty miscalibration caused by domain mismatch. Extensive experiments on both in-domain and cross-domain benchmarks demonstrate that the proposed approach consistently improves uncertainty reliability and speaker recognition robustness.

Problem

• uncertainty estimation
• uncertainty calibration
• domain shift
• speaker recognition
• speaker embedding

Innovation

• Uncertainty-Aware Speaker Modeling
• Inter- and Intra-Speaker Variability
• Uncertainty Calibration
• Domain Adaptation
• Speaker Embeddings