Local Concept Embeddings for Analysis of Concept Distributions in Vision DNN Feature Spaces

2023-11-24

AI Summary

Existing concept segmentation methods represent user-defined concepts (e.g., “car”) as single global vectors in deep neural network (DNN) latent spaces, failing to capture their underlying multimodal distributions. This limitation hinders fine-grained sub-concept discovery (e.g., “near car” vs. “far car”), exacerbates concept ambiguity (e.g., overlap between “bus” and “truck”), and impairs anomaly detection. To address this, we propose Local Concept Embedding (LoCE), the first framework to explicitly model the full distribution of a concept in DNN latent space via sample-level local embeddings—replacing global vector representations. LoCE integrates Gaussian mixture modeling, hierarchical clustering, and concept-level retrieval, and is architecture-agnostic, supporting both ViT and CNN backbones. Extensive experiments across three datasets and six models demonstrate that LoCE effectively uncovers sub-concept structures and concept confusion patterns, achieves significant gains in anomaly detection accuracy, and matches or exceeds global baseline performance in segmentation tasks.

Abstract

Insights into the learned latent representations are imperative for verifying deep neural networks (DNNs) in critical computer vision (CV) tasks. Therefore, state-of-the-art supervised Concept-based eXplainable Artificial Intelligence (C-XAI) methods associate user-defined concepts like ``car'' each with a single vector in the DNN latent space (concept embedding vector). In the case of concept segmentation, these linearly separate between activation map pixels belonging to a concept and those belonging to background. Existing methods for concept segmentation, however, fall short of capturing implicitly learned sub-concepts (e.g., the DNN might split car into ``proximate car'' and ``distant car''), and overlap of user-defined concepts (e.g., between ``bus'' and ``truck''). In other words, they do not capture the full distribution of concept representatives in latent space. For the first time, this work shows that these simplifications are frequently broken and that distribution information can be particularly useful for understanding DNN-learned notions of sub-concepts, concept confusion, and concept outliers. To allow exploration of learned concept distributions, we propose a novel local concept analysis framework. Instead of optimizing a single global concept vector on the complete dataset, it generates a local concept embedding (LoCE) vector for each individual sample. We use the distribution formed by LoCEs to explore the latent concept distribution by fitting Gaussian mixture models (GMMs), hierarchical clustering, and concept-level information retrieval and outlier detection. Despite its context sensitivity, our method's concept segmentation performance is competitive to global baselines. Analysis results are obtained on three datasets and six diverse vision DNN architectures, including vision transformers (ViTs).

Problem

• Analyzing sub-concepts and overlaps in DNN latent spaces
• Capturing full distribution of concept representatives
• Enhancing concept segmentation with local embeddings

Innovation

• Local concept embeddings for individual samples
• Gaussian mixture models for latent distribution
• Hierarchical clustering for concept exploration