Multi-Modal Monocular Endoscopic Depth and Pose Estimation with Edge-Guided Self-Supervision

2026-02-19

AI Summary

This work addresses the challenges of monocular depth and pose estimation in colonoscopy, where textureless surfaces, complex illumination, tissue deformation, and the absence of ground-truth annotations hinder performance. To overcome these issues, the authors propose PRISM, a self-supervised framework that integrates anatomical and photometric priors by jointly leveraging learned edge maps (e.g., from DexiNed or HED) and intrinsic image decomposition to disentangle shading and reflectance. The model exploits edge and shading cues to guide geometric learning, achieving state-of-the-art results across multiple real and synthetic datasets. Notably, this study is the first to reveal the critical influence of real video frame rates on self-supervised training efficacy and demonstrates that models trained on real data outperform those trained on ground-truth-annotated synthetic data.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Machine Learning: Unsupervised & Self-Supervised Learning
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Monocular depth and pose estimation play an important role in the development of colonoscopy-assisted navigation, as they enable improved screening by reducing blind spots, minimizing the risk of missed or recurrent lesions, and lowering the likelihood of incomplete examinations. However, this task remains challenging due to the presence of texture-less surfaces, complex illumination patterns, deformation, and a lack of in-vivo datasets with reliable ground truth. In this paper, we propose **PRISM** (Pose-Refinement with Intrinsic Shading and edge Maps), a self-supervised learning framework that leverages anatomical and illumination priors to guide geometric learning. Our approach uniquely incorporates edge detection and luminance decoupling for structural guidance. Specifically, edge maps are derived using a learning-based edge detector (e.g., DexiNed or HED) trained to capture thin and high-frequency boundaries, while luminance decoupling is obtained through an intrinsic decomposition module that separates shading and reflectance, enabling the model to exploit shading cues for depth estimation. Experimental results on multiple real and synthetic datasets demonstrate state-of-the-art performance. We further conduct a thorough ablation study on training data selection to establish best practices for pose and depth estimation in colonoscopy. This analysis yields two practical insights: (1) self-supervised training on real-world data outperforms supervised training on realistic phantom data, underscoring the superiority of domain realism over ground truth availability; and (2) video frame rate is an extremely important factor for model performance, where dataset-specific video frame sampling is necessary for generating high quality training data.

Problem

• monocular depth estimation
• pose estimation
• colonoscopy
• self-supervision
• multi-modal

Innovation

• edge-guided self-supervision
• intrinsic image decomposition
• monocular depth estimation
• colonoscopy navigation
• pose refinement