DCBM: Data-Efficient Visual Concept Bottleneck Models

2024-12-16

AI Summary

Existing concept bottleneck models (CBMs) rely on large-scale textual corpora or image datasets to generate concepts, limiting their interpretability and generalization under data-scarce conditions. This paper proposes a text-free, data-efficient visual concept generation framework: it leverages segmentation/detection foundation models to localize multi-granularity image regions as interpretable concepts, integrates Grad-CAM-based attribution analysis, and adopts a concept-driven two-stage prediction architecture. Its core contribution is the first realization of image region–based concept construction—without reliance on pretrained language models or massive textual supervision. The method maintains high accuracy and strong interpretability under low-shot settings, supports fine-grained classification and out-of-distribution generalization, provides pixel-level concept localization via saliency heatmaps, and enables rapid adaptation to novel domains—significantly enhancing cross-domain transferability.

Technology Category

• Computer Vision: Large Vision Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Multimodal Learning

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Concept Bottleneck Models (CBMs) enhance the interpretability of neural networks by basing predictions on human-understandable concepts. However, current CBMs typically rely on concept sets extracted from large language models or extensive image corpora, limiting their effectiveness in data-sparse scenarios. We propose Data-efficient CBMs (DCBMs), which reduce the need for large sample sizes during concept generation while preserving interpretability. DCBMs define concepts as image regions detected by segmentation or detection foundation models, allowing each image to generate multiple concepts across different granularities. This removes reliance on textual descriptions and large-scale pre-training, making DCBMs applicable for fine-grained classification and out-of-distribution tasks. Attribution analysis using Grad-CAM demonstrates that DCBMs deliver visual concepts that can be localized in test images. By leveraging dataset-specific concepts instead of predefined ones, DCBMs enhance adaptability to new domains.

Problem

• Reduces large sample size need in concept generation
• Enhances interpretability without extensive pre-training
• Improves adaptability to new domains effectively

Innovation

• Uses segmentation for concept generation
• Reduces dependency on large datasets
• Improves adaptability across new domains