Development and evaluation of CADe systems in low-prevalence setting: The RARE25 challenge for early detection of Barrett's neoplasia

2026-04-13

AI Summary

This study addresses the insufficient clinical evaluation of computer-aided detection (CADe) systems for early neoplasia in Barrett’s esophagus under low-prevalence screening settings by launching the RARE25 challenge. It establishes the first standardized benchmark tailored to low-prevalence scenarios, providing large-scale, prevalence-aware public training and hidden test sets, along with operating-point-specific metrics that prioritize high sensitivity while accounting for disease prevalence. Participating methods employed fully supervised classification frameworks, integrating diverse deep learning architectures, pretraining strategies, and model calibration techniques. Results revealed that although several models demonstrated strong discriminative performance, their positive predictive values remained consistently low, highlighting a pronounced sensitivity to prevalence shifts and a tendency to overestimate real-world clinical utility. These findings underscore the critical need for more robust and clinically practical CADe systems.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Other Foundations of Computer Vision
• Search and Optimization: Evaluation and Analysis

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Computer-aided detection (CADe) of early neoplasia in Barrett's esophagus is a low-prevalence surveillance problem in which clinically relevant findings are rare. Although many CADe systems report strong performance on balanced or enriched datasets, their behavior under realistic prevalence remains insufficiently characterized. The RARE25 challenge addresses this gap by introducing a large-scale, prevalence-aware benchmark for neoplasia detection. It includes a public training set and a hidden test set reflecting real-world incidence. Methods were evaluated using operating-point-specific metrics emphasizing high sensitivity and accounting for prevalence. Eleven teams from seven countries submitted approaches using diverse architectures, pretraining, ensembling, and calibration strategies. While several methods achieved strong discriminative performance, positive predictive values remained low, highlighting the difficulty of low-prevalence detection and the risk of overestimating clinical utility when prevalence is ignored. All methods relied on fully supervised classification despite the dominance of normal findings, indicating a lack of prevalence-agnostic approaches such as anomaly detection or one-class learning. By releasing a public dataset and a reproducible evaluation framework, RARE25 aims to support the development of CADe systems robust to prevalence shift and suitable for clinical surveillance workflows.

Problem

• Barrett's neoplasia
• low-prevalence detection
• computer-aided detection
• clinical surveillance
• prevalence shift

Innovation

• low-prevalence detection
• computer-aided detection (CADe)
• Barrett's neoplasia
• prevalence-aware benchmark
• RARE25 challenge