Credal Concept Bottleneck Models for Epistemic-Aleatoric Uncertainty Decomposition

2026-04-27

AI Summary

Traditional concept bottleneck models rely on point estimates that fail to distinguish between epistemic and aleatoric uncertainty, limiting their interpretability and practical utility. This work proposes CREDENCE, a novel framework that structurally decomposes these two uncertainty sources within concept bottleneck models for the first time: epistemic uncertainty is quantified via disagreement among multiple concept prediction heads, while aleatoric uncertainty is modeled using annotator disagreement as supervision, yielding probabilistic intervals in the form of prediction sets. The approach enables actionable decision policies—such as automated processing, human review, or active abstention—based on uncertainty-aware outputs. Experiments demonstrate that epistemic uncertainty strongly correlates with prediction errors, and aleatoric uncertainty accurately captures annotation disagreement, outperforming conventional evaluation metrics that rely solely on error rates.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Concept Bottleneck Models (CBMs) predict through human-interpretable concepts, but they typically output point concept probabilities that conflate epistemic uncertainty (reducible model underspecification) with aleatoric uncertainty (irreducible input ambiguity). This makes concept-level uncertainty hard to interpret and, more importantly, hard to act upon. We introduce CREDENCE (Credal Ensemble Concept Estimation), a CBM framework that decomposes concept uncertainty by construction. CREDENCE represents each concept as a credal prediction (a probability interval), derives epistemic uncertainty from disagreement across diverse concept heads, and estimates aleatoric uncertainty via a dedicated ambiguity output trained to match annotator disagreement when available. The resulting signals support prescriptive decisions: automate low-uncertainty cases, prioritize data collection for high-epistemic cases, route high-aleatoric cases to human review, and abstain when both are high. Across several tasks, we show that epistemic uncertainty is positively associated with prediction errors, whereas aleatoric uncertainty closely tracks annotator disagreement, providing guidance beyond error correlation. Our implementation is available at the following link: https://github.com/Tankiit/Credal_Sets/tree/ensemble-credal-cbm

Problem

• Concept Bottleneck Models
• epistemic uncertainty
• aleatoric uncertainty
• uncertainty decomposition
• credal sets

Innovation

• Credal Sets
• Uncertainty Decomposition
• Concept Bottleneck Models
• Epistemic Uncertainty
• Aleatoric Uncertainty