Hybrid(Transformer+CNN)-based Polyp Segmentation

2025-08-07

AI Summary

Colon polyp segmentation is highly challenging due to morphological variability, ambiguous boundaries (e.g., occlusion by fluid or mucosal folds), and endoscopic artifacts (e.g., specular highlights and motion blur). To address these issues, we propose a novel hybrid architecture integrating CNNs and Transformers, featuring a boundary-aware attention mechanism and a multi-level feature fusion strategy. This design jointly models fine-grained local details and long-range spatial dependencies, thereby enhancing robustness against complex artifacts. Extensive experiments on multiple public benchmarks demonstrate state-of-the-art performance: a Dice score of 0.9849 and a recall of 0.9555—surpassing existing methods significantly. Notably, our approach achieves superior accuracy in scenarios with ill-defined boundaries and severe interference, validating its effectiveness for clinically demanding segmentation tasks.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Data transparency and provenance
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

Colonoscopy is still the main method of detection and segmentation of colonic polyps, and recent advancements in deep learning networks such as U-Net, ResUNet, Swin-UNet, and PraNet have made outstanding performance in polyp segmentation. Yet, the problem is extremely challenging due to high variation in size, shape, endoscopy types, lighting, imaging protocols, and ill-defined boundaries (fluid, folds) of the polyps, rendering accurate segmentation a challenging and problematic task. To address these critical challenges in polyp segmentation, we introduce a hybrid (Transformer + CNN) model that is crafted to enhance robustness against evolving polyp characteristics. Our hybrid architecture demonstrates superior performance over existing solutions, particularly in addressing two critical challenges: (1) accurate segmentation of polyps with ill-defined margins through boundary-aware attention mechanisms, and (2) robust feature extraction in the presence of common endoscopic artifacts, including specular highlights, motion blur, and fluid occlusions. Quantitative evaluations reveal significant improvements in segmentation accuracy (Recall improved by 1.76%, i.e., 0.9555, accuracy improved by 0.07%, i.e., 0.9849) and artifact resilience compared to state-of-the-art polyp segmentation methods.

Problem

• Addressing polyp segmentation challenges due to size and shape variations
• Improving segmentation accuracy for polyps with ill-defined boundaries
• Enhancing robustness against endoscopic artifacts like blur and occlusions

Innovation

• Hybrid Transformer-CNN model for polyp segmentation
• Boundary-aware attention for ill-defined margins
• Robust feature extraction against endoscopic artifacts