Marginalized Bundle Adjustment: Multi-View Camera Pose from Monocular Depth Estimates

2026-02-21

AI Summary

Monocular depth estimation (MDE) provides dense geometric priors but suffers from high error variance, limiting its utility in multi-view 3D reconstruction (Structure-from-Motion, SfM). This work proposes Marginalized Bundle Adjustment (MBA), which for the first time integrates modern RANSAC-inspired ideas into a bundle adjustment framework by explicitly modeling and marginalizing MDE depth errors to effectively suppress their uncertainty. By fully leveraging the potential of dense depth maps, MBA achieves state-of-the-art or competitive performance in camera pose estimation and SfM across diverse scales—from just a few frames to thousands of images—significantly enhancing the practical value of MDE in multi-view 3D vision.

Technology Category

• Intelligent Robots: State Estimation
• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Structure-from-Motion (SfM) is a fundamental 3D vision task for recovering camera parameters and scene geometry from multi-view images. While recent deep learning advances enable accurate Monocular Depth Estimation (MDE) from single images without depending on camera motion, integrating MDE into SfM remains a challenge. Unlike conventional triangulated sparse point clouds, MDE produces dense depth maps with significantly higher error variance. Inspired by modern RANSAC estimators, we propose Marginalized Bundle Adjustment (MBA) to mitigate MDE error variance leveraging its density. With MBA, we show that MDE depth maps are sufficiently accurate to yield SoTA or competitive results in SfM and camera relocalization tasks. Through extensive evaluations, we demonstrate consistently robust performance across varying scales, ranging from few-frame setups to large multi-view systems with thousands of images. Our method highlights the significant potential of MDE in multi-view 3D vision.

Problem

• Monocular Depth Estimation
• Structure-from-Motion
• Bundle Adjustment
• Camera Pose Estimation
• Multi-View 3D Vision

Innovation

• Marginalized Bundle Adjustment
• Monocular Depth Estimation
• Structure-from-Motion
• dense depth maps
• camera relocalization