Restyling Unsupervised Concept Based Interpretable Networks with Generative Models

2024-07-01

arXiv.org

AI Summary

This work addresses the challenges of concept visualization and semantic interpretation in unsupervised concept learning—particularly the low fidelity, poor intuitiveness, and weak cross-sample consistency exhibited by existing methods on large-scale image datasets. To this end, we propose a novel interpretability framework. Our core methodological innovation is the first construction of a differentiable mapping from concept features to the latent space of pretrained generative models (e.g., StyleGAN), enabling efficient and faithful alignment while preserving concept fidelity. Furthermore, we introduce a generative explanation paradigm based on concept activation interpolation, supporting interactive semantic editing and inversion-based visualization. Evaluated across multiple large-scale image recognition benchmarks, our approach maintains high predictive accuracy while significantly improving concept reconstruction fidelity, explanation faithfulness, and cross-sample consistency.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Transparent, Interpretable, Explainable ML
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Developing inherently interpretable models for prediction has gained prominence in recent years. A subclass of these models, wherein the interpretable network relies on learning high-level concepts, are valued because of closeness of concept representations to human communication. However, the visualization and understanding of the learnt unsupervised dictionary of concepts encounters major limitations, especially for large-scale images. We propose here a novel method that relies on mapping the concept features to the latent space of a pretrained generative model. The use of a generative model enables high quality visualization, and lays out an intuitive and interactive procedure for better interpretation of the learnt concepts by imputing concept activations and visualizing generated modifications. Furthermore, leveraging pretrained generative models has the additional advantage of making the training of the system more efficient. We quantitatively ascertain the efficacy of our method in terms of accuracy of the interpretable prediction network, fidelity of reconstruction, as well as faithfulness and consistency of learnt concepts. The experiments are conducted on multiple image recognition benchmarks for large-scale images. Project page available at https://jayneelparekh.github.io/VisCoIN_project_page/

Problem

• Improving visualization of unsupervised concept-based interpretable networks
• Enhancing interpretation of large-scale image concepts using generative models
• Increasing training efficiency with pretrained generative models

Innovation

• Mapping concept features to generative model latent space
• Using generative models for high-quality concept visualization
• Leveraging pretrained models for efficient system training