Gaussian Belief Propagation Network for Depth Completion

2026-01-29

AI Summary

This work addresses the challenge of modeling sparse and irregular depth inputs in depth completion, particularly the performance degradation under high sparsity. To this end, the authors propose an end-to-end framework that integrates deep learning with probabilistic graphical models. The method adaptively learns a non-local graph structure to dynamically construct a scene-specific Markov random field, and incorporates Gaussian belief propagation (GBP) with a serial-parallel hybrid message-passing mechanism to effectively capture long-range dependencies while improving inference efficiency. The proposed model achieves state-of-the-art performance on the NYUv2 and KITTI benchmarks, demonstrating strong robustness and generalization across varying sparsity levels, sampling patterns, and cross-dataset settings.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Diffusion Models for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Depth completion aims to predict a dense depth map from a color image with sparse depth measurements. Although deep learning methods have achieved state-of-the-art (SOTA), effectively handling the sparse and irregular nature of input depth data in deep networks remains a significant challenge, often limiting performance, especially under high sparsity. To overcome this limitation, we introduce the Gaussian Belief Propagation Network (GBPN), a novel hybrid framework synergistically integrating deep learning with probabilistic graphical models for end-to-end depth completion. Specifically, a scene-specific Markov Random Field (MRF) is dynamically constructed by the Graphical Model Construction Network (GMCN), and then inferred via Gaussian Belief Propagation (GBP) to yield the dense depth distribution. Crucially, the GMCN learns to construct not only the data-dependent potentials of MRF but also its structure by predicting adaptive non-local edges, enabling the capture of complex, long-range spatial dependencies. Furthermore, we enhance GBP with a serial \&parallel message passing scheme, designed for effective information propagation, particularly from sparse measurements. Extensive experiments demonstrate that GBPN achieves SOTA performance on the NYUv2 and KITTI benchmarks. Evaluations across varying sparsity levels, sparsity patterns, and datasets highlight GBPN's superior performance, notable robustness, and generalizable capability.

Problem

• Depth Completion
• Sparse Depth
• Irregular Data
• High Sparsity
• Dense Depth Prediction

Innovation

• Gaussian Belief Propagation
• Depth Completion
• Markov Random Field
• Non-local Dependencies
• Hybrid Deep Learning