ClinOCR-Bench: A Comprehensive Clinical Scanned Document Dataset for Optical Character Recognition Model Evaluation

2026-07-03

AI Summary

Existing clinical OCR research lacks a publicly available, realistic benchmark dataset that encompasses common scanning artifacts, hindering systematic evaluation of model robustness and accuracy in electronic health records. To address this gap, this work introduces ClinOCR-Bench—a publicly released dataset comprising 384 real-world scanned images spanning six representative clinical document types, fully preserving typical scanning artifacts. The dataset employs a template-aware train/test split strategy to prevent data leakage and contains no protected health information, thereby achieving an unprecedented balance of diversity, realism, and reproducibility. Baseline evaluations are provided using both open-source and closed-source vision-language models. The dataset and code are publicly available on GitHub, establishing a standardized evaluation platform for clinical OCR research.

Abstract

Extracting textual information from scanned medical documents, such as external laboratory reports and manually filled forms, has been a major challenge in modern electronic health records (EHRs). Recent advancements in vision language models (VLMs) have shown great promise over traditional OCR tools. However, at this point, most clinical OCR studies were conducted on private, institutional data. To our knowledge, there are few publicly available datasets for evaluating OCR models in the clinical domain. Furthermore, common scanning artifacts that undermine OCR performance are not reflected in those datasets, leaving a systematic evaluation unfeasible. Therefore, we release a publicly available, realistic-looking OCR benchmark dataset, ClinOCR-Bench, with 384 scanned images across 6 subsets: Normal, Handwriting, Poor Quality, Rotation, Tables, and Mix-artifacts. ClinOCR-Bench features: 1) diverse document types and layouts, 2) full coverage of common EHR scan artifacts, 3) protected health information-free, 4) template-aware train/test split, and 5) adequate sample size for OCR benchmarking. Baseline OCR performance was evaluated using state-of-the-art open-weight and proprietary VLMs. The dataset and documentation are available on GitHub (https://github.com/ClinOCR-Bench/ClinOCR-Bench).

Problem

• clinical OCR
• scanned medical documents
• OCR benchmark
• EHR artifacts
• public dataset

Innovation

• Clinical OCR
• Scanned Document Dataset
• Vision Language Models
• Benchmarking
• EHR Artifacts