TrialBench: Multi-Modal Artificial Intelligence-Ready Clinical Trial Datasets

2024-06-30

arXiv.org

AI Summary

Clinical trials frequently incur substantial resource waste due to high failure rates and prolonged development timelines; AI adoption remains hindered by the absence of standardized multimodal data and systematically defined prediction tasks. To address this, we introduce the first AI-ready multimodal dataset specifically designed for clinical trial planning, integrating drug molecules (SMILES), disease codes (ICD), clinical text, and structured trial features. It supports eight critical prediction tasks—including trial duration, patient dropout rate, incidence of serious adverse events, and regulatory approval outcomes—spanning the entire trial lifecycle. We formally propose, validate via expert clinician annotation and task alignment, and benchmark a comprehensive AI prediction taxonomy. A unified evaluation protocol and baseline models (XGBoost, Transformer) are established. The dataset, evaluation metrics, and code are fully open-sourced, substantially lowering barriers for AI researchers and already enabling multiple studies on trial simulation and design optimization.

Technology Category

• Machine Learning: Multimodal Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Clinical trials are pivotal for developing new medical treatments, yet they typically pose some risks such as patient mortality, adverse events, and enrollment failure that waste immense efforts spanning over a decade. Applying artificial intelligence (AI) to forecast or simulate key events in clinical trials holds great potential for providing insights to guide trial designs. However, complex data collection and question definition requiring medical expertise and a deep understanding of trial designs have hindered the involvement of AI thus far. This paper tackles these challenges by presenting a comprehensive suite of meticulously curated AIready datasets covering multi-modal data (e.g., drug molecule, disease code, text, categorical/numerical features) and 8 crucial prediction challenges in clinical trial design, encompassing prediction of trial duration, patient dropout rate, serious adverse event, mortality rate, trial approval outcome, trial failure reason, drug dose finding, design of eligibility criteria. Furthermore, we provide basic validation methods for each task to ensure the datasets' usability and reliability. We anticipate that the availability of such open-access datasets will catalyze the development of advanced AI approaches for clinical trial design, ultimately advancing clinical trial research and accelerating medical solution development. The curated dataset, metrics, and basic models are publicly available at https://github.com/ML2Health/ML2ClinicalTrials/tree/main/AI4Trial.

Problem

• Predicting key events in clinical trials using AI
• Overcoming complex data collection and medical expertise barriers
• Providing multi-modal datasets for clinical trial design challenges

Innovation

• Multi-modal AI-ready datasets for clinical trials
• Covers 8 key prediction challenges in trials
• Includes validation methods for dataset reliability