Dynamic Distribution-Aware Uncertainty Tracking in Vision-Language Representation Learning

2026-08-09

AI Summary

This work addresses the limitation of existing posterior uncertainty quantification methods, which struggle to adapt to dynamic shifts in test distributions and remain confined to failure modes observed in the source domain. To overcome this, we propose a dynamic distribution-aware uncertainty quantification framework that models the distributional characteristics of vision-language model embedding spaces using Gaussian mixture models. By dynamically eliciting distributional evidence during both training and inference, our approach enables adaptive estimation of predictive uncertainty. This framework breaks away from conventional static mapping paradigms, allowing uncertainty measures to adjust in real time according to the evolving test data distribution. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art uncertainty quantification approaches across multiple benchmarks.

Abstract

Uncertainty Quantification (UQ) aims to measure the reliability of model predictions, serving as a critical safeguard for deploying Vision-Language Models (VLMs) in safety-critical scenarios. Post-hoc approaches are widely adopted due to their lightweight nature, mapping the outputs of VLMs to uncertainty measures through learnable modules or inductive summarization. However, Post-hoc approaches remain inherently confined to fitting the failure patterns of the source domain, ignoring the dynamic nature of test distributions. To address this challenge, we propose a Dynamic Distribution-Aware Uncertainty Quantification framework (DDA-UQ) that shifts the paradigm from static mapping to a dynamic distribution-aware process. During training, we leverage a Gaussian Mixture Model to model the VVLMs'embedding space and extract distributional evidence, thereby dynamically deriving uncertainty estimates. During inference, the design dynamically responds to changes in the data distribution. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods.

Problem

• Uncertainty Quantification
• Vision-Language Models
• Distribution Shift
• Post-hoc Methods
• Dynamic Distribution

Innovation

• Dynamic Distribution-Aware
• Uncertainty Quantification
• Vision-Language Models
• Gaussian Mixture Model
• Post-hoc Calibration