Solve the Missing First Step: Can VLMs Standardize Raw Heterogeneous Medical Data?

2026-07-06

AI Summary

This study addresses a critical gap in medical AI research, which typically assumes standardized input data while overlooking the heterogeneity and fragmentation of raw clinical data in real-world settings. To bridge this gap, the authors propose an end-to-end framework for standardizing raw medical data, integrating modules for data format identification, image preprocessing, text extraction, and structured image-text pair generation. They introduce MDS-Bench, the first benchmark specifically designed to evaluate such standardization pipelines across diverse clinical scenarios and data formats, leveraging vision-language models (VLMs) for systematic assessment. Experimental results reveal that even the current state-of-the-art VLM, Gemini 1.5 Flash, achieves only a 48.6% end-to-end success rate on this task, underscoring data standardization as a fundamental bottleneck and a vital direction for advancing clinically deployable medical AI systems.

Abstract

As vision-language models (VLMs) are increasingly applied to medical AI, existing benchmarks mainly focus on evaluating their diagnosis ability over given medical images and texts, implicitly assuming that standardized medical images, texts or question-answer pairs are already prepared. However, this assumption does not hold when we apply VLMs in real clinical practice, where medical data is often raw, heterogeneous, and fragmented across different sources. In this paper, we study this missing step, i.e., raw medical data standardization. Specifically, models are given raw dataset folders and evaluated on their ability to identify source formats, convert raw medical images into VLM-compatible visual inputs, extract relevant textual information, and organize the results into structured image-text pairs. To construct this Medical Data Standardization Benchmark (MDS-Bench), we manually annotate 1,939 raw medical data standardization tasks covering diverse clinical practice, radiology modalities, annotation formats, and directory layouts. Extensive experiments show that even the best performing VLMs, i.e., Gemini 3 Flash, achieve only 48.6% end-to-end success rate. Our research highlights raw medical data standardization as a critical bottleneck for medical AI diagnosis in real practice.

Problem

• medical data standardization
• vision-language models
• raw heterogeneous data
• clinical practice
• data preprocessing

Innovation

• medical data standardization
• vision-language models
• heterogeneous medical data
• MDS-Bench
• real-world clinical AI