SAGE: An Expert-Annotated South Asian GI Endoscopy Dataset for Multimodal Learning and Hallucination Analysis

2026-06-20

AI Summary

This study addresses the significant geographic bias in existing AI models for gastrointestinal endoscopy diagnosis, which stems from the underrepresentation of South Asian populations. To bridge this gap, the authors introduce and publicly release the first multimodal endoscopic dataset tailored to South Asian patients, comprising 1,300 images, expert-annotated textual descriptions, multilabel classifications, and 14,726 question-answer pairs, thereby supporting tasks in image classification, image captioning, and visual question answering. Notably, the dataset incorporates a hallucination labeling mechanism to enhance reliability. Benchmark evaluations reveal a striking performance degradation: specialized multiclass models exhibit an average 58% drop in accuracy, while leading large multimodal models achieve GREEN scores of only 0.308 and 0.410 on anatomical landmark identification and abnormality detection tasks, respectively, underscoring a critical cross-regional generalization bottleneck.

Abstract

Gastrointestinal cancers represent a growing health burden in the South Asian region, driven largely by rapid changes in socio-economic conditions & lifestyle habits. However, early diagnosis of such malignancies remains a significant challenge, largely due to a lack of modern equipment, lack of financial support, and a scarcity of GI experts. AI-assisted diagnosis & report generation, show great promise in alleviating this problem by providing low-skill manpower the technical expertise to perform diagnosis. However, almost all open-source, publicly available datasets are predominantly collected from the European region, with no representation from the South Asian region. The lack of open-source GI datasets from diverse geographic regions has made it difficult to assess whether population bias is present in existing models, and to develop geographically inclusive AI tools for automated GI diagnosis. To address this gap, we introduce SAGE: An Expert-Annotated South Asian GI Endoscopy dataset for image captioning, multi-label classification, and visual question answering (VQA) tasks. It consists of 1,300 images, their captions along with hallucination tag, 18 labels and 14,726 question-answer pairs making it well-suited for diverse range of tasks including classification, benchmarking, and fine-tuning large multimodal models (LMMs). We further conducted benchmarking of multi-class classifiers on the effect of population shift in GI imaging AI tasks, and contemporary LMMs on their performance. Our study reveals that task-specific models, such as multi-class classification models, suffer the most, with an average performance drop of 58% when evaluated on the South Asian dataset. For contemporary LMMs, benchmarking reveals a substantial drop in the average GREEN score for anatomical landmark detection (0.308) and abnormality detection (0.410).

Problem

• gastrointestinal cancer
• population bias
• South Asian dataset
• AI-assisted diagnosis
• geographic inclusivity

Innovation

• multimodal learning
• hallucination analysis
• geographic bias
• GI endoscopy dataset
• large multimodal models