A Mutual Information Lower Bound for Multimodal Regression Active Learning

2026-05-14

AI Summary

Existing active learning methods for continuous regression struggle to effectively characterize epistemic uncertainty in multimodal prediction settings. This work proposes the Two-Index framework, which explicitly disentangles epistemic from aleatoric uncertainty and introduces, for the first time, a mutual information–based measure of epistemic uncertainty tailored to multimodal regression active learning. The acquisition function is constructed via the mutual information between model outputs and an epistemic index, yielding a closed-form lower bound termed MI-LB. Theoretically, this measure is shown to vanish asymptotically with increasing data, ensuring it captures only uncertainty reducible through observation. Empirical results demonstrate that MI-LB consistently matches or outperforms existing baselines on multimodal regression benchmarks, whereas geometric and Fisher-based approaches succeed only when inputs already encode multimodal structure and otherwise fail.

Abstract

Active learning for continuous regression has lacked an acquisition function that targets epistemic uncertainty when the predictive distribution is multimodal: variance misses modal disagreement, and information-theoretic targets like BALD are designed for discrete outputs. We introduce a Two-Index framework that makes this separation explicit: one stochastic index selects among competing model hypotheses (epistemic source), while a second governs within-hypothesis randomness (aleatoric source). An entropy decomposition within the framework identifies the mutual information between the output and the epistemic index as a principled acquisition objective, and we prove this quantity vanishes as the model is trained on growing datasets, confirming that it captures exactly the uncertainty data can resolve. Because this mutual information is intractable for continuous outputs, we derive the Mutual Information Lower Bound (MI-LB) acquisition function, a closed-form approximation for Mixture Density Network ensembles. On benchmarks featuring multimodal systems, MI-LB matches or beats every baseline evaluated and is the only method to do so consistently -- geometric and Fisher-based baselines compete only when the input space already encodes the multimodality, and collapse otherwise.

Problem

• active learning
• multimodal regression
• epistemic uncertainty
• acquisition function
• mutual information

Innovation

• mutual information lower bound
• multimodal regression
• active learning
• epistemic uncertainty
• mixture density networks