MedConcept: Unsupervised Concept Discovery for Interpretability in Medical VLMs

2026-04-13

AI Summary

This work addresses the limited clinical trustworthiness of medical vision-language models (VLMs) due to their black-box nature, which hinders the provision of interpretable and reusable concept-level explanations. The authors propose MedConcept, a framework that, for the first time, discovers sparse neuron-level concepts from pretrained medical VLMs in an unsupervised manner and generates radiology-report-like natural language summaries through concept-to-text mapping. Innovatively, a frozen medical large language model is introduced as an external evaluator to establish a quantitative semantic validation protocol, categorizing discovered concepts into Aligned, Unaligned, and Uncertain types. This study establishes the first benchmark for evaluating interpretability in medical VLMs, significantly enhancing model transparency and clinical utility.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

While medical Vision-Language models (VLMs) achieve strong performance on tasks such as tumor or organ segmentation and diagnosis prediction, their opaque latent representations limit clinical trust and the ability to explain predictions. Interpretability of these multimodal representations are therefore essential for the trustworthy clinical deployment of pretrained medical VLMs. However, current interpretability methods, such as gradient- or attention-based visualizations, are often limited to specific tasks such as classification. Moreover, they do not provide concept-level explanations derived from shared pretrained representations that can be reused across downstream tasks. We introduce MedConcept, a framework that uncovers latent medical concepts in a fully unsupervised manner and grounds them in clinically verifiable textual semantics. MedConcept identifies sparse neuron-level concept activations from pretrained VLM representations and translates them into pseudo-report-style summaries, enabling physician-level inspection of internal model reasoning. To address the lack of quantitative evaluation in concept-based interpretability, we introduce a quantitative semantic verification protocol that leverages an independent pretrained medical LLM as a frozen external evaluator to assess concept alignment with radiology reports. We define three concept scores, Aligned, Unaligned, and Uncertain, to quantify semantic support, contradiction, or ambiguity relative to radiology reports and use them exclusively for post hoc evaluation. These scores provide a quantitative baseline for assessing interpretability in medical VLMs. All codes, prompt and data to be released on acceptance. Ke

Problem

• interpretability
• medical VLMs
• concept discovery
• unsupervised learning
• clinical trust

Innovation

• unsupervised concept discovery
• medical vision-language models
• concept-level interpretability
• semantic verification
• explainable AI