CDPR: Cross-modal Diffusion with Polarization for Reliable Monocular Depth Estimation

2026-04-13

AI Summary

Monocular depth estimation suffers from limited performance in textureless, transparent, or specular regions due to insufficient RGB cues. This work proposes the first cross-modal diffusion framework that integrates RGB and polarization information, mapping multimodal inputs into a shared latent space via a pretrained VAE and enabling dynamic fusion through a confidence-aware gating mechanism. The method innovatively embeds physics-driven polarization priors into the diffusion model and introduces a learnable strategy for multimodal fusion. Experiments demonstrate that the approach significantly outperforms RGB-only baselines on both synthetic and real-world datasets, with pronounced improvements in challenging regions while maintaining competitive accuracy in conventional scenes. Furthermore, the model generalizes effectively to other dense prediction tasks such as surface normal estimation.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Security and Privacy: Data transparency and provenance

Abstract

Monocular depth estimation is a fundamental yet challenging task in computer vision, especially under complex conditions such as textureless surfaces, transparency, and specular reflections. Recent diffusion-based approaches have significantly advanced performance by reformulating depth prediction as a denoising process in the latent space. However, existing methods rely solely on RGB inputs, which often lack sufficient cues in challenging regions. In this work, we present CDPR - Cross-modal Diffusion with Polarization for Reliable Monocular Depth Estimation - a novel diffusion-based framework that integrates physically grounded polarization priors to enhance estimation robustness. Specifically, we encode both RGB and polarization (AoLP/DoLP) images into a shared latent space via a pre-trained Variational Autoencoder (VAE), and dynamically fuse multi-modal information through a learnable confidence-aware gating mechanism. This fusion module adaptively suppresses noisy signals in polarization inputs while preserving informative cues, particularly around reflective or transparent surfaces, and provides the integrated latent representation for subsequent monocular depth estimation. Beyond depth estimation, we further verify that our framework can be easily generalized to surface normal prediction with minimal modification, showcasing its scalability to general polarization-guided dense prediction tasks. Experiments on both synthetic and real-world datasets validate that CDPR significantly outperforms RGB-only baselines in challenging regions while maintaining competitive performance in standard scenes.

Problem

• monocular depth estimation
• polarization
• cross-modal fusion
• specular reflections
• transparent surfaces

Innovation

• cross-modal diffusion
• polarization cues
• monocular depth estimation
• confidence-aware fusion
• dense prediction