PVTAdpNet: Polyp Segmentation using Pyramid vision transformer with a novel Adapter block

2025-09-28

AI Summary

To address the high polyp miss-rate in colonoscopy caused by significant morphological and scale variations, this paper proposes a U-shaped segmentation network integrating a Pyramid Vision Transformer (PVT) encoder with a novel adapter module. We innovatively design adapter-based skip connections and residual structures, augmented with Squeeze-and-Excitation channel-wise attention, to substantially enhance multi-scale feature fusion and gradient flow stability—thereby improving cross-domain generalization. Leveraging PVT as the backbone encoder, the architecture combines lightweight adapters with a U-Net decoder for efficient dense prediction. Extensive evaluations demonstrate state-of-the-art performance: mean Dice score of 0.8851 and mean IoU of 0.8167 across multiple benchmark and cross-domain datasets. Moreover, on the PolypGen dataset, the model achieves both real-time inference speed and high-precision polyp segmentation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Security and Privacy: Privacy-enhancing technologies

Abstract

Colorectal cancer ranks among the most common and deadly cancers, emphasizing the need for effective early detection and treatment. To address the limitations of traditional colonoscopy, including high miss rates due to polyp variability, we introduce the Pyramid Vision Transformer Adapter Residual Network (PVTAdpNet). This model integrates a U-Net-style encoder-decoder structure with a Pyramid Vision Transformer backbone, novel residual blocks, and adapter-based skip connections. The design enhances feature extraction, dense prediction, and gradient flow, supported by squeeze-and-excitation attention for improved channel-wise feature refinement. PVTAdpNet achieves real-time, accurate polyp segmentation, demonstrating superior performance on benchmark datasets with high mDice and mIoU scores, making it highly suitable for clinical applications. PVTAdpNet obtains a high Dice coefficient of 0.8851 and a mean Intersection over Union (mIoU) of 0.8167 on out-of-distribution polyp datasets. Evaluation of the PolypGen dataset demonstrates PVTAdpNet's capability for real-time, accurate performance within familiar distributions. The source code of our network is available at https://github.com/ayousefinejad/PVTAdpNet.git

Problem

• Enhancing polyp segmentation accuracy in colonoscopy images
• Overcoming polyp variability limitations in colorectal cancer detection
• Achieving real-time performance for clinical polyp segmentation applications

Innovation

• Uses Pyramid Vision Transformer backbone for feature extraction
• Integrates novel adapter blocks for enhanced skip connections
• Employs squeeze-and-excitation attention for feature refinement