Semi-supervised Concept Bottleneck Models

2024-06-27

arXiv.org

AI Summary

Existing concept bottleneck models (CBMs) heavily rely on high-quality, labor-intensive human concept annotations and frequently suffer from misalignment between concept saliency and input saliency. To address the challenge of scarce labeled data, this paper proposes a semi-supervised concept bottleneck model (SSCBM)—the first to integrate semi-supervised learning into the CBM framework. SSCBM introduces a concept-level pseudo-labeling strategy and a concept-space alignment loss to enforce consistency constraints at the concept level over unlabeled samples. By jointly optimizing on both labeled and unlabeled data, SSCBM achieves 93.19% concept accuracy and 75.51% prediction accuracy using only 20% of the labeled data, approaching fully supervised performance (96.39% / 79.82%). This significantly reduces expert annotation effort while mitigating concept–input misalignment.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Computer Vision: Large Vision Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Concept Bottleneck Models (CBMs) have garnered increasing attention due to their ability to provide concept-based explanations for black-box deep learning models while achieving high final prediction accuracy using human-like concepts. However, the training of current CBMs heavily relies on the accuracy and richness of annotated concepts in the dataset. These concept labels are typically provided by experts, which can be costly and require significant resources and effort. Additionally, concept saliency maps frequently misalign with input saliency maps, causing concept predictions to correspond to irrelevant input features - an issue related to annotation alignment. To address these limitations, we propose a new framework called SSCBM (Semi-supervised Concept Bottleneck Model). Our SSCBM is suitable for practical situations where annotated data is scarce. By leveraging joint training on both labeled and unlabeled data and aligning the unlabeled data at the concept level, we effectively solve these issues. We proposed a strategy to generate pseudo labels and an alignment loss. Experiments demonstrate that our SSCBM is both effective and efficient. With only 20% labeled data, we achieved 93.19% (96.39% in a fully supervised setting) concept accuracy and 75.51% (79.82% in a fully supervised setting) prediction accuracy.

Problem

• Reduces reliance on costly expert-annotated concept labels
• Addresses misalignment between concept and input saliency maps
• Improves performance with limited labeled data using semi-supervised learning

Innovation

• Semi-supervised learning with labeled and unlabeled data
• Concept-level alignment for improved accuracy
• Pseudo label generation and alignment loss strategy