Digging Deeper: Learning Multi-Level Concept Hierarchies

2026-03-10

AI Summary

This work addresses the limitations of existing concept-based models, which rely on fine-grained annotations and treat concepts as flat, independent units, thereby hindering the construction of interpretable, hierarchical concept structures. To overcome this, the authors propose Multi-Level Concept Segmentation (MLCS) and Deep Hierarchical Concept Embedding Models (Deep-HiCEMs), which require only coarse-grained top-level supervision to automatically discover multi-layered, human-interpretable concept hierarchies. The framework supports concept interventions across abstraction levels and successfully uncovers novel, explainable concepts absent from training data across multiple benchmarks. While maintaining high predictive accuracy, the method significantly enhances task performance through test-time interventions, marking the first approach capable of automatically constructing a multi-granular concept system from coarse-grained labels alone.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Segmentation

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Although concept-based models promise interpretability by explaining predictions with human-understandable concepts, they typically rely on exhaustive annotations and treat concepts as flat and independent. To circumvent this, recent work has introduced Hierarchical Concept Embedding Models (HiCEMs) to explicitly model concept relationships, and Concept Splitting to discover sub-concepts using only coarse annotations. However, both HiCEMs and Concept Splitting are restricted to shallow hierarchies. We overcome this limitation with Multi-Level Concept Splitting (MLCS), which discovers multi-level concept hierarchies from only top-level supervision, and Deep-HiCEMs, an architecture that represents these discovered hierarchies and enables interventions at multiple levels of abstraction. Experiments across multiple datasets show that MLCS discovers human-interpretable concepts absent during training and that Deep-HiCEMs maintain high accuracy while supporting test-time concept interventions that can improve task performance.

Problem

• concept hierarchy
• interpretability
• hierarchical representation
• concept-based models
• multi-level concepts

Innovation

• Multi-Level Concept Splitting
• Deep-HiCEMs
• concept hierarchy
• interpretability
• concept intervention