🤖 AI Summary
This study addresses the challenge of accurate detection, segmentation, and endoscopy–histology joint classification of colorectal polyps by proposing a three-stage hierarchical deep learning framework. The first stage employs EfficientNetV2-M for polyp type and morphology classification; the second stage utilizes UNet++ for high-precision segmentation and recommends resection strategies; and the final stage refines adenoma subtyping through transfer learning. Innovatively integrating a task-progressive architecture, task-specific loss functions (Focal Loss and Dice+BCE), and cross-stage knowledge transfer, the framework achieves device-agnostic fully automated analysis. Evaluated on PolypGen, Kvasir-SEG, and CVC-ClinicDB datasets, it attains an AUC of 0.99 and mAP@50 of 94.4%, matching or surpassing current state-of-the-art methods, and has been deployed as a publicly accessible web application.
📝 Abstract
Colorectal cancer (CRC) remains one of the leading causes of cancer-related mortality worldwide, predominantly arising from precancerous polyps. Accurate detection, segmentation, and endoscopic and histological classification of colorectal polyps are crucial for timely clinical intervention. In this study, we present PolypVision, a three-stage hierarchical deep learning framework that sequentially performs: (Stage 1) binary classification of polyps as adenomatous or hyperplastic, with simultaneous Paris and JNet classification, using EfficientNetV2-M with Focal Loss; (Stage 2) polyp segmentation with recommended resection method using a UNet++ decoder with the Stage 1 backbone as encoder, optimized with Dice and BCE losses; and (Stage 3) adenoma subtype classification (tubular, tubulovillous, villous) using EfficientNetV2-M with transfer learning from Stage 2. Evaluated on three public datasets -- PolypGen, Kvasir-SEG, and CVC-ClinicDB -- PolypVision achieves an AUC of approximately 0.99 for frame classification and a detection mAP@50 of 94.4% on Kvasir-SEG, outperforming or matching state-of-the-art methods. Gradient-weighted Class Activation Maps (Grad-CAM) confirm that the model attends to clinically relevant lesion features. The framework is device-independent, operating across diverse endoscopic imaging systems without hardware-specific adaptation. These results demonstrate that a hierarchical, transfer-learning-driven pipeline with task-specific loss functions offers a robust, device-independent, and clinically meaningful approach to automated colorectal polyp analysis. PolypVision is freely available as a web application at https://polypvision.com, a DataBioX initiative, with a free usage tier open to all users.