Big Batch Bayesian Active Learning by Considering Predictive Probabilities

2025-01-14

AI Summary

BatchBALD conflates epistemic and aleatoric uncertainty, limiting scalability and performance in large-scale Bayesian active learning. Method: We propose PureBALD—the first acquisition function that models epistemic uncertainty *exclusively* via predictive probabilities, fully decoupling it from aleatoric uncertainty. It introduces an efficient, scalable batch acquisition objective that avoids the combinatorial bottleneck of joint sample evaluation inherent in BatchBALD. Built upon Bayesian neural networks, Monte Carlo sampling, and mutual information approximation, PureBALD optimizes both predictive entropy and joint epistemic uncertainty. Contribution/Results: Across multiple image and text benchmarks, PureBALD consistently outperforms BatchBALD—achieving 3–5× higher query efficiency, supporting batch sizes up to 1,000, faster convergence, and superior generalization.

Technology Category

• Machine Learning: Active Learning
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

We observe that BatchBALD, a popular acquisition function for batch Bayesian active learning for classification, can conflate epistemic and aleatoric uncertainty, leading to suboptimal performance. Motivated by this observation, we propose to focus on the predictive probabilities, which only exhibit epistemic uncertainty. The result is an acquisition function that not only performs better, but is also faster to evaluate, allowing for larger batches than before.

Problem

• BatchBALD
• Uncertainty
• Bayesian Active Learning

Innovation

• Bayesian Active Learning
• BatchBALD Improvement
• Large-scale Dataset Handling