Intercept Cancer: Cancer Pre-Screening with Large Scale Healthcare Foundation Models

2025-05-30

AI Summary

Current cancer screening relies on costly, invasive procedures with limited accessibility, leading to substantial underdiagnosis of early-stage cases. To address this, we propose CATCH-FM—the first foundational EHR model pretrained exclusively on raw ICD code sequences (2.4B parameters)—introducing the first medical code sequence scaling law and a clinical-text-free modeling paradigm for robust, cross-system and cross-population risk prediction. Leveraging temporal EHR representation learning and supervised fine-tuning on expert-annotated cohorts, CATCH-FM achieves 60% sensitivity, 99% specificity, and 99% negative predictive value in a retrospective evaluation of 30,000 cases. It ranks first on the EHRSHOT pancreatic cancer few-shot benchmark, significantly outperforming both general-purpose and domain-specific large language models.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Cancer screening, leading to early detection, saves lives. Unfortunately, existing screening techniques require expensive and intrusive medical procedures, not globally available, resulting in too many lost would-be-saved lives. We present CATCH-FM, CATch Cancer early with Healthcare Foundation Models, a cancer pre-screening methodology that identifies high-risk patients for further screening solely based on their historical medical records. With millions of electronic healthcare records (EHR), we establish the scaling law of EHR foundation models pretrained on medical code sequences, pretrain compute-optimal foundation models of up to 2.4 billion parameters, and finetune them on clinician-curated cancer risk prediction cohorts. In our retrospective evaluation comprising of thirty thousand patients, CATCH-FM achieved strong efficacy (60% sensitivity) with low risk (99% specificity and Negative Predictive Value), outperforming feature-based tree models as well as general and medical large language models by large margins. Despite significant demographic, healthcare system, and EHR coding differences, CATCH-FM achieves state-of-the-art pancreatic cancer risk prediction on the EHRSHOT few-shot leaderboard, outperforming EHR foundation models pretrained using on-site patient data. Our analysis demonstrates the robustness of CATCH-FM in various patient distributions, the benefits of operating in the ICD code space, and its ability to capture non-trivial cancer risk factors. Our code will be open-sourced.

Problem

• Developing non-invasive cancer pre-screening using healthcare foundation models
• Improving early cancer detection with scalable EHR-based risk prediction
• Overcoming limitations of costly traditional screening methods globally

Innovation

• Uses large-scale healthcare foundation models
• Analyzes historical medical records for risk
• Achieves high sensitivity and specificity