Towards automatizing detection and quantification of intestinal metaplasia: a multi-expert comparative study

2025-09-02

AI Summary

Gastric cancer risk assessment commonly relies on pathologists’ subjective visual estimation of intestinal metaplasia (IM) percentage, resulting in poor reproducibility and inter-observer agreement. To address this, we propose the first deep learning–based framework for automatic IM quantification and systematically evaluate its consistency against multiple expert pathologists in IM grading. Our method employs a multi-scale convolutional architecture and is assessed using F1-score, area under the curve (AUC), and Fleiss’s kappa. The optimal model achieves F1 = 0.80 and AUC = 0.91—significantly surpassing the moderate inter-expert agreement (κ = 0.61–0.75) and revealing substantial variability in manual assessment (low κ between model and individual experts). This study provides the first empirical evidence that AI-based IM quantification attains both high accuracy and superior reproducibility, establishing a robust, standardized technical foundation for gastric cancer risk stratification.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Current gastric cancer risk systems are prone to errors since they evaluate a visual estimation of intestinal metaplasia percentages to assign a risk. This study presents an automated method to detect and quantify intestinal metaplasia using deep learning models as well as a comparative analysis with visual estimations of three experienced pathologists. Gastric samples were collected from two different cohorts: 149 asymptomatic volunteers from a region with a high prevalence of GCa in Colombia and 56 patients from a third-level hospital. Deep learning models were selected and trained to classify intestinal metaplasia, and predictions were used to estimate the percentage of intestinal metaplasia and assign the risk score. Results were compared with independent blinded assessments performed by three experienced pathologists. The best-performing deep learning architecture classified intestinal metaplasia with F1-Score of 0.80 +- 0.01 and AUC of 0.91 +- 0.01. Among pathologists, inter-observer agreement by a Fleiss's Kappa score ranged from 0.61 to 0.75. In comparison, agreement between the pathologists and the best-performing model ranged from 0.37 to 0.54. Deep learning models show potential to detect and quantify the percentage of intestinal metaplasia with greater precision and reproducibility than experienced pathologists. Likewise, estimated risk shows high inter-observer variability when visually assigning the intestinal metaplasia percentage.

Problem

• Automating detection and quantification of intestinal metaplasia
• Reducing errors in gastric cancer risk assessment systems
• Comparing deep learning models with pathologists' visual estimations

Innovation

• Deep learning models automate intestinal metaplasia detection
• Quantifies metaplasia percentage for gastric cancer risk
• Compares AI performance against expert pathologists' assessments