Hierarchical Concept Embedding&Pursuit for Interpretable Image Classification

2026-02-11

AI Summary

This work addresses the limitation of existing interpretable image classification methods, which neglect the semantic hierarchy of concepts during concept recovery, leading to explanations misaligned with human cognition. To bridge this gap, the study explicitly incorporates the hierarchical structure of concepts into the sparse coding process for the first time. It constructs hierarchical concept embeddings in the latent space of a vision-language model and enforces semantic coherence along root-to-leaf paths in the concept tree through hierarchical sparse coding and path-based semantic constraints. Experimental results demonstrate that the proposed approach significantly improves both concept precision and recall while maintaining competitive classification accuracy, with particularly pronounced gains in few-shot settings.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Multimodal Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Interpretable-by-design models are gaining traction in computer vision because they provide faithful explanations for their predictions. In image classification, these models typically recover human-interpretable concepts from an image and use them for classification. Sparse concept recovery methods leverage the latent space of vision-language models to represent image embeddings as a sparse combination of concept embeddings. However, because such methods ignore the hierarchical structure of concepts, they can produce correct predictions with explanations that are inconsistent with the hierarchy. In this work, we propose Hierarchical Concept Embedding \&Pursuit (HCEP), a framework that induces a hierarchy of concept embeddings in the latent space and uses hierarchical sparse coding to recover the concepts present in an image. Given a hierarchy of semantic concepts, we construct a corresponding hierarchy of concept embeddings and, assuming the correct concepts for an image form a rooted path in the hierarchy, derive desirable conditions for identifying them in the embedded space. We show that hierarchical sparse coding reliably recovers hierarchical concept embeddings, whereas vanilla sparse coding fails. Our experiments on real-world datasets demonstrate that HCEP outperforms baselines in concept precision and recall while maintaining competitive classification accuracy. Moreover, when the number of samples is limited, HCEP achieves superior classification accuracy and concept recovery. These results show that incorporating hierarchical structures into sparse coding yields more reliable and interpretable image classification models.

Problem

• interpretable image classification
• concept hierarchy
• sparse concept recovery
• hierarchical structure
• vision-language models

Innovation

• hierarchical concept embedding
• hierarchical sparse coding
• interpretable image classification
• vision-language models
• concept recovery