MP-SfM: Monocular Surface Priors for Robust Structure-from-Motion

2025-04-28

AI Summary

Existing monocular Structure-from-Motion (SfM) methods suffer from poor robustness under extreme viewpoint changes, low image overlap, small parallax, or highly symmetric scenes—limiting their practicality for non-expert users. To address this, we propose a robust monocular SfM framework that tightly integrates single-image depth and surface normal priors with multi-view geometric constraints. Our method introduces an uncertainty-aware prior fusion mechanism that accommodates diverse monocular models and exhibits strong resilience to prior estimation errors. Notably, it achieves reliable 3D reconstruction of challenging indoor scenes from only a few images—a capability not previously demonstrated in monocular SfM. Experiments show that our approach significantly outperforms state-of-the-art SfM methods under extreme viewpoints while maintaining competitive accuracy on standard benchmarks. Moreover, it effectively mitigates false correspondences induced by scene symmetry. The implementation is publicly available.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

While Structure-from-Motion (SfM) has seen much progress over the years, state-of-the-art systems are prone to failure when facing extreme viewpoint changes in low-overlap, low-parallax or high-symmetry scenarios. Because capturing images that avoid these pitfalls is challenging, this severely limits the wider use of SfM, especially by non-expert users. We overcome these limitations by augmenting the classical SfM paradigm with monocular depth and normal priors inferred by deep neural networks. Thanks to a tight integration of monocular and multi-view constraints, our approach significantly outperforms existing ones under extreme viewpoint changes, while maintaining strong performance in standard conditions. We also show that monocular priors can help reject faulty associations due to symmetries, which is a long-standing problem for SfM. This makes our approach the first capable of reliably reconstructing challenging indoor environments from few images. Through principled uncertainty propagation, it is robust to errors in the priors, can handle priors inferred by different models with little tuning, and will thus easily benefit from future progress in monocular depth and normal estimation. Our code is publicly available at https://github.com/cvg/mpsfm.

Problem

• Enhances SfM robustness in extreme viewpoint changes
• Addresses faulty associations due to symmetries in SfM
• Improves reconstruction of challenging indoor environments

Innovation

• Integrates monocular depth and normal priors
• Combines monocular and multi-view constraints tightly
• Uses uncertainty propagation for robust error handling