Eye-Tracking, Mouse Tracking, Stimulus Tracking,and Decision-Making Datasets in Digital Pathology

2025-10-28

AI Summary

In digital pathology diagnosis, pathologists achieve only ~70% accuracy on gigapixel whole-slide images (WSIs), and inter-observer agreement remains low—even with dual review—primarily due to the absence of high-ecological-validity behavioral mechanism data. To address this, we introduce PathoGaze1.0, the first multimodal digital pathology behavioral dataset: it captures synchronized eye movements (171,909 fixations, 263,320 saccades), mouse interactions (1,867,362 events), viewport navigation, and stimulus timestamps from 19 pathologists analyzing 397 WSIs under realistic diagnostic conditions—totaling 18.69 hours of behavioral data. Data acquisition and millisecond-precision multimodal synchronization are enabled by the PTAH platform, balancing temporal accuracy and ecological validity. All data and code are fully open-sourced, establishing a foundational resource for diagnostic error attribution, human-AI collaborative modeling, and rigorous evaluation of AI-assisted pathology systems.

Technology Category

• Computer Vision: Multi-modal Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

Interpretation of giga-pixel whole-slide images (WSIs) is an important but difficult task for pathologists. Their diagnostic accuracy is estimated to average around 70%. Adding a second pathologist does not substantially improve decision consistency. The field lacks adequate behavioral data to explain diagnostic errors and inconsistencies. To fill in this gap, we present PathoGaze1.0, a comprehensive behavioral dataset capturing the dynamic visual search and decision-making processes of the full diagnostic workflow during cancer diagnosis. The dataset comprises 18.69 hours of eye-tracking, mouse interaction, stimulus tracking, viewport navigation, and diagnostic decision data (EMSVD) collected from 19 pathologists interpreting 397 WSIs. The data collection process emphasizes ecological validity through an application-grounded testbed, called PTAH. In total, we recorded 171,909 fixations, 263,320 saccades, and 1,867,362 mouse interaction events. In addition, such data could also be used to improve the training of both pathologists and AI systems that might support human experts. All experiments were preregistered at https://osf.io/hj9a7, and the complete dataset along with analysis code is available at https://go.osu.edu/pathogaze.

Problem

• Explaining diagnostic errors and inconsistencies in pathology
• Capturing visual search and decision-making in cancer diagnosis
• Addressing lack of behavioral data for pathology workflow analysis

Innovation

• Developed PathoGaze1.0 behavioral dataset for pathology
• Integrated eye-tracking and mouse interaction data collection
• Created PTAH testbed for ecologically valid experiments