Exploring Radiologists' Expectations of Explainable Machine Learning Models in Medical Image Analysis

2026-04-13

AI Summary

Current explainable machine learning models struggle to gain acceptance among radiologists and integrate into clinical workflows due to a lack of clinical perspective. This study systematically investigates radiologists’ needs, preferred tasks, and deployment preferences for explainable AI through structured questionnaires and qualitative interviews across varying levels of experience and subspecialties. By integrating clinical task analysis with human factors engineering principles, the work uniquely articulates practice-oriented design requirements from the clinical user’s viewpoint. The resulting set of clinically grounded guidelines for developing explainable AI effectively addresses the prevailing gap in existing approaches—which often prioritize technical implementation over clinical validation—and substantially enhances the practical utility and potential for clinical integration of such models.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Machine Learning: Transparent, Interpretable, Explainable ML
• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In spite of the strong performance of machine learning (ML) models in radiology, they have not been widely accepted by radiologists, limiting clinical integration. A key reason is the lack of explainability, which ensures that model predictions are understandable and verifiable by clinicians. Several methods and tools have been proposed to improve explainability, but most reflect developers' perspectives and lack systematic clinical validation. In this work, we gathered insights from radiologists with varying experience and specialties into explainable ML requirements through a structured questionnaire. They also highlighted key clinical tasks where ML could be most beneficial and how it might be deployed. Based on their input, we propose guidelines for designing and developing explainable ML models in radiology. These guidelines can help researchers develop clinically useful models, facilitating integration into radiology practice as a supportive tool.

Problem

• explainable machine learning
• radiology
• clinical integration
• model interpretability
• medical image analysis

Innovation

• Explainable AI
• Radiology
• Clinical Validation
• Human-Centered Design
• Medical Image Analysis