A Temporal Convolutional Network-Based Approach and a Benchmark Dataset for Colonoscopy Video Temporal Segmentation

2025-02-05

AI Summary

To address the need for automated reporting in colonoscopy video analysis, this paper introduces REAL-Colon—the first open-source, end-to-end temporal segmentation benchmark for colonoscopy videos, featuring frame-level annotations across nine classes (five anatomical landmarks and four procedural phases). We propose ColonTCN, a lightweight temporal convolutional network incorporating customized dilated convolutions to effectively model long-range temporal dependencies. To ensure robust generalization and clinical relevance, we devise a novel dual k-fold cross-center evaluation protocol. Experiments demonstrate that ColonTCN achieves state-of-the-art frame-level classification accuracy under both k-fold settings, with significantly fewer parameters than existing methods. Ablation studies confirm the critical contribution of the temporal modeling module to performance gains. This work establishes a high-quality benchmark, an efficient architecture, and a rigorous, clinically grounded evaluation framework for intelligent colonoscopy analysis.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Machine Learning: Evaluation and Analysis
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Following recent advancements in computer-aided detection and diagnosis systems for colonoscopy, the automated reporting of colonoscopy procedures is set to further revolutionize clinical practice. A crucial yet underexplored aspect in the development of these systems is the creation of computer vision models capable of autonomously segmenting full-procedure colonoscopy videos into anatomical sections and procedural phases. In this work, we aim to create the first open-access dataset for this task and propose a state-of-the-art approach, benchmarked against competitive models. We annotated the publicly available REAL-Colon dataset, consisting of 2.7 million frames from 60 complete colonoscopy videos, with frame-level labels for anatomical locations and colonoscopy phases across nine categories. We then present ColonTCN, a learning-based architecture that employs custom temporal convolutional blocks designed to efficiently capture long temporal dependencies for the temporal segmentation of colonoscopy videos. We also propose a dual k-fold cross-validation evaluation protocol for this benchmark, which includes model assessment on unseen, multi-center data.ColonTCN achieves state-of-the-art performance in classification accuracy while maintaining a low parameter count when evaluated using the two proposed k-fold cross-validation settings, outperforming competitive models. We report ablation studies to provide insights into the challenges of this task and highlight the benefits of the custom temporal convolutional blocks, which enhance learning and improve model efficiency. We believe that the proposed open-access benchmark and the ColonTCN approach represent a significant advancement in the temporal segmentation of colonoscopy procedures, fostering further open-access research to address this clinical need.

Problem

• Automated temporal segmentation of colonoscopy videos.
• Creation of an open-access dataset for colonoscopy analysis.
• Development of a Temporal Convolutional Network for video segmentation.

Innovation

• Temporal Convolutional Network
• Open-access benchmark dataset
• Dual k-fold cross-validation