SHADeS: Self-supervised Monocular Depth Estimation Through Non-Lambertian Image Decomposition

2025-02-18

AI Summary

Monocular depth estimation in colonoscopy is highly challenging due to strong specular reflections and complex illumination. To address this, we propose a self-supervised, end-to-end framework that—uniquely in monocular depth estimation—explicitly models non-Lambertian reflectance by decoupling specular reflection as an independent component, thereby disentangling lighting from geometry. Our method jointly estimates depth, surface normals, albedo, shading, and specular reflectance. The learning objective integrates multiple physically grounded constraints: differentiable physics-based rendering, photometric consistency, normal smoothness, and a sparsity prior on specular highlights. Evaluated on the real-world Hyper-Kvasir dataset and the synthetic C3VD dataset, our approach reduces mean depth error by 19.3% and improves depth accuracy within specular regions by 41.7%, significantly outperforming state-of-the-art baselines including IID and MonoVIT. Moreover, the estimated geometry robustly supports downstream tasks such as anatomical localization and polyp recognition.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Knowledge Representation and Reasoning: Nonmonotonic Reasoning

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Data transparency and provenance

Abstract

Purpose: Visual 3D scene reconstruction can support colonoscopy navigation. It can help in recognising which portions of the colon have been visualised and characterising the size and shape of polyps. This is still a very challenging problem due to complex illumination variations, including abundant specular reflections. We investigate how to effectively decouple light and depth in this problem. Methods: We introduce a self-supervised model that simultaneously characterises the shape and lighting of the visualised colonoscopy scene. Our model estimates shading, albedo, depth, and specularities (SHADeS) from single images. Unlike previous approaches (IID), we use a non-Lambertian model that treats specular reflections as a separate light component. The implementation of our method is available at https://github.com/RemaDaher/SHADeS. Results: We demonstrate on real colonoscopy images (Hyper Kvasir) that previous models for light decomposition (IID) and depth estimation (MonoVIT, ModoDepth2) are negatively affected by specularities. In contrast, SHADeS can simultaneously produce light decomposition and depth maps that are robust to specular regions. We also perform a quantitative comparison on phantom data (C3VD) where we further demonstrate the robustness of our model. Conclusion: Modelling specular reflections improves depth estimation in colonoscopy. We propose an effective self-supervised approach that uses this insight to jointly estimate light decomposition and depth. Light decomposition has the potential to help with other problems, such as place recognition within the colon.

Problem

• Decouple light and depth
• Improve colonoscopy navigation
• Handle specular reflections

Innovation

• Self-supervised monocular depth estimation
• Non-Lambertian image decomposition
• Robust specular reflection handling