On dataset transferability in medical image classification

2024-12-28

AI Summary

Existing general-purpose image transformations and transferability assessment methods perform poorly for cross-domain transfer of pretrained models—particularly from natural to medical imaging—and lack reliable benchmarks. Method: We propose the first transferability metric that jointly quantifies feature quality and gradient sensitivity, pioneering the incorporation of gradient signals into medical image transfer evaluation and challenging the implicit assumption that the target dataset is inherently the optimal source. We further construct the first ground-truth benchmark dataset for medical image transfer performance. Results: Experiments demonstrate that our metric significantly outperforms state-of-the-art baselines in both intra-domain (source-to-source) and cross-domain (natural-to-medical) transfer scenarios. Moreover, it uncovers key determinants of medical image transferability and characterizes the systematic degradation of cross-domain performance.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Interpretability, Explainability, and Transparency
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

Current transferability estimation methods designed for natural image datasets are often suboptimal in medical image classification. These methods primarily focus on estimating the suitability of pre-trained source model features for a target dataset, which can lead to unrealistic predictions, such as suggesting that the target dataset is the best source for itself. To address this, we propose a novel transferability metric that combines feature quality with gradients to evaluate both the suitability and adaptability of source model features for target tasks. We evaluate our approach in two new scenarios: source dataset transferability for medical image classification and cross-domain transferability. Our results show that our method outperforms existing transferability metrics in both settings. We also provide insight into the factors influencing transfer performance in medical image classification, as well as the dynamics of cross-domain transfer from natural to medical images. Additionally, we provide ground-truth transfer performance benchmarking results to encourage further research into transferability estimation for medical image classification. Our code and experiments are available at https://github.com/DovileDo/transferability-in-medical-imaging.

Problem

• Medical Image Classification
• Feature Adaptability
• Accuracy Improvement

Innovation

• Medical Image Classification
• Adaptive Image Transformation
• Cross-Domain Data Adaptation