Extracting Uncertainty Estimates from Mixtures of Experts for Semantic Segmentation

2025-09-05

AI Summary

This work addresses the challenge of extracting reliable uncertainty estimates from Mixture-of-Experts (MoE) models for semantic segmentation—particularly in safety-critical traffic scenarios. We propose a plug-and-play method that requires no architectural modification: it leverages the gating network to dynamically weight expert outputs and jointly quantifies predictive uncertainty via entropy, mutual information, and expert variance, while using gating entropy to characterize routing uncertainty. Experiments on A2D2 and Cityscapes demonstrate that our approach significantly improves conditional calibration on out-of-distribution data, outperforming conventional ensembles; a simple gating mechanism facilitates better calibration of routing uncertainty; and increasing the number of experts further enhances uncertainty estimation reliability. Our core contribution is the first systematic validation that MoE’s inherent structure offers a natural advantage for uncertainty modeling in semantic segmentation, coupled with a lightweight, efficient, and high-fidelity uncertainty extraction paradigm.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Reasoning under Uncertainty: Uncertainty Representations
• Planning, Routing, and Scheduling: Planning under Uncertainty

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Estimating accurate and well-calibrated predictive uncertainty is important for enhancing the reliability of computer vision models, especially in safety-critical applications like traffic scene perception. While ensemble methods are commonly used to quantify uncertainty by combining multiple models, a mixture of experts (MoE) offers an efficient alternative by leveraging a gating network to dynamically weight expert predictions based on the input. Building on the promising use of MoEs for semantic segmentation in our previous works, we show that well-calibrated predictive uncertainty estimates can be extracted from MoEs without architectural modifications. We investigate three methods to extract predictive uncertainty estimates: predictive entropy, mutual information, and expert variance. We evaluate these methods for an MoE with two experts trained on a semantical split of the A2D2 dataset. Our results show that MoEs yield more reliable uncertainty estimates than ensembles in terms of conditional correctness metrics under out-of-distribution (OOD) data. Additionally, we evaluate routing uncertainty computed via gate entropy and find that simple gating mechanisms lead to better calibration of routing uncertainty estimates than more complex classwise gates. Finally, our experiments on the Cityscapes dataset suggest that increasing the number of experts can further enhance uncertainty calibration. Our code is available at https://github.com/KASTEL-MobilityLab/mixtures-of-experts/.

Problem

• Extracting uncertainty estimates from mixtures of experts
• Improving reliability of semantic segmentation models
• Evaluating uncertainty calibration under out-of-distribution data

Innovation

• Mixture of Experts for semantic segmentation
• Extracting uncertainty via entropy and variance
• Dynamic gating improves uncertainty calibration