Case-Aware Medical Image Classification with Multimodal Knowledge Graphs and Reliability-Guided Refinement

2026-05-21

AI Summary

This work addresses the limitations of existing medical image classification methods that rely primarily on isolated visual features and struggle to integrate information from similar cases and external knowledge for interpretable diagnosis. The authors propose a case-driven multimodal knowledge graph framework that constructs a graph structure by retrieving clinically similar cases and leverages graph attention networks together with bidirectional cross-modal attention mechanisms to enable effective knowledge propagation and injection. To further refine decision-making, the approach incorporates a confidence-calibrated case reliability assessment and an adaptive fusion strategy. Extensive experiments demonstrate that the proposed method significantly outperforms strong baseline models across multiple medical imaging datasets, while ablation studies confirm the contribution of each component and provide case-level interpretability for diagnostic decisions.

Abstract

Deep learning has brought significant progress to medical image classification, yet most existing methods still rely on isolated visual evidence and cannot effectively leverage similar cases or external knowledge. In clinical practice, diagnosis is typically supported by historical similar cases and their associated symptoms. To simulate this diagnostic process, we propose a framework that performs case-aware reasoning using multimodal knowledge graphs for explainable medical image diagnosis. Given an input image, our method constructs a multimodal knowledge graph from adaptively retrieved similar cases, enabling more effective utilization of related samples. We further introduce a knowledge propagation and injection mechanism, where an image-centric Graph Attention Network propagates knowledge semantics to obtain case-based features, followed by a bidirectional cross-modal attention mechanism that injects these features into visual representations for cross-modal alignment. To mitigate noisy retrieval, we design a confidence-calibrated decision refinement scheme that estimates the reliability of each retrieved case by jointly considering prediction confidence and sample similarity, adaptively adjusting its contribution to the final prediction and providing interpretable case-level evidence. Extensive experiments on multiple medical imaging datasets show that our approach consistently outperforms strong baselines, and ablation studies validate the effectiveness of each component. The source code is publicly available at https://anonymous.4open.science/r/MKG-CARE-8B7B.

Problem

• medical image classification
• case-aware reasoning
• multimodal knowledge graphs
• clinical diagnosis
• knowledge integration

Innovation

• multimodal knowledge graph
• case-aware reasoning
• graph attention network
• cross-modal alignment
• reliability-guided refinement