Intervening in Black Box: Concept Bottleneck Model for Enhancing Human Neural Network Mutual Understanding

2025-06-28

AI Summary

Increasing complexity of deep learning models degrades interpretability, and existing explanation methods are largely post-hoc, offering no mechanism to intervene in model behavior. Method: We propose CBM-HNMU—a Concept Bottleneck Model-based framework that (i) decomposes black-box decision logic via concept representations; (ii) automatically identifies and localizes harmful concepts through global gradient contribution analysis; and (iii) performs reverse optimization via concept replacement and knowledge distillation—without altering the original architecture. Contribution/Results: CBM-HNMU enables human-interpretable, bidirectional interaction and achieves automatic identification, correction, and performance enhancement of erroneous internal concepts. Evaluated across multiple datasets on both CNNs and Transformers, it improves average accuracy by 1.03% (up to 2.64%) while simultaneously enhancing interpretability and generalization.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Interpretability, Explainability, and Transparency
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Recent advances in deep learning have led to increasingly complex models with deeper layers and more parameters, reducing interpretability and making their decisions harder to understand. While many methods explain black-box reasoning, most lack effective interventions or only operate at sample-level without modifying the model itself. To address this, we propose the Concept Bottleneck Model for Enhancing Human-Neural Network Mutual Understanding (CBM-HNMU). CBM-HNMU leverages the Concept Bottleneck Model (CBM) as an interpretable framework to approximate black-box reasoning and communicate conceptual understanding. Detrimental concepts are automatically identified and refined (removed/replaced) based on global gradient contributions. The modified CBM then distills corrected knowledge back into the black-box model, enhancing both interpretability and accuracy. We evaluate CBM-HNMU on various CNN and transformer-based models across Flower-102, CIFAR-10, CIFAR-100, FGVC-Aircraft, and CUB-200, achieving a maximum accuracy improvement of 2.64% and a maximum increase in average accuracy across 1.03%. Source code is available at: https://github.com/XiGuaBo/CBM-HNMU.

Problem

• Enhancing interpretability of complex deep learning models
• Enabling effective intervention in black-box model reasoning
• Improving model accuracy through concept refinement

Innovation

• Uses Concept Bottleneck Model for interpretability
• Refines concepts via global gradient contributions
• Distills corrected knowledge into black-box models