MetaFE-DE: Learning Meta Feature Embedding for Depth Estimation from Monocular Endoscopic Images

2025-02-05

AI Summary

Monocular endoscopic depth estimation faces significant challenges due to large soft-tissue deformations and highly variable illumination, while existing RGB-to-depth regression methods suffer from limited interpretability and accuracy. To address these issues, we propose MetaFE (Meta Feature Embedding), a novel representation paradigm that models tissues and instruments as shared latent-space features jointly decodable into either RGB or depth maps. We further design a two-stage self-supervised framework integrating diffusion-based temporal modeling, cross-normalized spatial feature alignment, and brightness-calibrated depth decoding. Our approach is the first to achieve physically grounded disentangled representation learning with multi-constraint cooperative optimization. Evaluated on diverse multi-source endoscopic datasets, it substantially outperforms state-of-the-art methods—reducing mean depth error by 12.6% and demonstrating markedly improved cross-domain generalization.

Technology Category

• Computer Vision: Multi-modal Vision
• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Multimodal Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Depth estimation from monocular endoscopic images presents significant challenges due to the complexity of endoscopic surgery, such as irregular shapes of human soft tissues, as well as variations in lighting conditions. Existing methods primarily estimate the depth information from RGB images directly, and often surffer the limited interpretability and accuracy. Given that RGB and depth images are two views of the same endoscopic surgery scene, in this paper, we introduce a novel concept referred as ``meta feature embedding (MetaFE)", in which the physical entities (e.g., tissues and surgical instruments) of endoscopic surgery are represented using the shared features that can be alternatively decoded into RGB or depth image. With this concept, we propose a two-stage self-supervised learning paradigm for the monocular endoscopic depth estimation. In the first stage, we propose a temporal representation learner using diffusion models, which are aligned with the spatial information through the cross normalization to construct the MetaFE. In the second stage, self-supervised monocular depth estimation with the brightness calibration is applied to decode the meta features into the depth image. Extensive evaluation on diverse endoscopic datasets demonstrates that our approach outperforms the state-of-the-art method in depth estimation, achieving superior accuracy and generalization. The source code will be publicly available.

Problem

• Monocular endoscopic depth estimation
• Meta feature embedding
• Self-supervised learning paradigm

Innovation

• Meta feature embedding concept
• Two-stage self-supervised learning
• Diffusion models for temporal representation