ClassifyViStA:WCE Classification with Visual understanding through Segmentation and Attention

2024-12-24

AI Summary

To address the challenges of high computational cost, low accuracy, and poor interpretability in gastrointestinal bleeding detection from wireless capsule endoscopy (WCE) videos, this paper proposes an end-to-end dual-path AI framework integrating an implicit attention branch and an explicit segmentation branch for joint hemorrhage/non-hemorrhage classification and pixel-level localization. Methodologically, we introduce Soft Non-Maximum Suppression (Soft NMS) to refine YOLOv8-based hemorrhage region detection and leverage segmentation masks to generate clinician-level decision visualizations. The architecture combines an ensemble classifier (ResNet-18/VGG-16), attention-guided feature enhancement, and a semantic segmentation module. Evaluated on a WCE dataset, our method achieves 98.2% classification accuracy and 96.7% intersection-over-union (IoU) for hemorrhage regions—substantially reducing reading time while delivering clinically trustworthy, attribution-based explanations.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

Gastrointestinal (GI) bleeding is a serious medical condition that presents significant diagnostic challenges, particularly in settings with limited access to healthcare resources. Wireless Capsule Endoscopy (WCE) has emerged as a powerful diagnostic tool for visualizing the GI tract, but it requires time-consuming manual analysis by experienced gastroenterologists, which is prone to human error and inefficient given the increasing number of patients.To address this challenge, we propose ClassifyViStA, an AI-based framework designed for the automated detection and classification of bleeding and non-bleeding frames from WCE videos. The model consists of a standard classification path, augmented by two specialized branches: an implicit attention branch and a segmentation branch.The attention branch focuses on the bleeding regions, while the segmentation branch generates accurate segmentation masks, which are used for classification and interpretability. The model is built upon an ensemble of ResNet18 and VGG16 architectures to enhance classification performance. For the bleeding region detection, we implement a Soft Non-Maximum Suppression (Soft NMS) approach with YOLOv8, which improves the handling of overlapping bounding boxes, resulting in more accurate and nuanced detections.The system's interpretability is enhanced by using the segmentation masks to explain the classification results, offering insights into the decision-making process similar to the way a gastroenterologist identifies bleeding regions. Our approach not only automates the detection of GI bleeding but also provides an interpretable solution that can ease the burden on healthcare professionals and improve diagnostic efficiency. Our code is available at ClassifyViStA.

Problem

• Wireless Capsule Endoscopy
• Gastrointestinal Bleeding
• Medical Resource Scarcity

Innovation

• AI-assisted diagnosis
• Bleeding detection
• Wireless Capsule Endoscopy