Mono3R: Exploiting Monocular Cues for Geometric 3D Reconstruction

2025-04-18

AI Summary

To address the poor robustness of multi-view 3D reconstruction in weak-texture and low-light scenes—caused by insufficient matching cues—this paper proposes a monocular geometry-guided fine-grained reconstruction framework. Our method is the first to embed monocular depth and pose priors into the multi-view matching pipeline in an end-to-end manner, eliminating reliance on traditional iterative optimization and enabling feed-forward high-accuracy reconstruction. Key technical contributions include: (i) Transformer-based multi-view feature matching; (ii) differentiable geometric consistency constraints; and (iii) a cross-modal feature fusion distillation mechanism. Evaluated on multiple benchmarks, our approach achieves a 32% reduction in average relative rotation error for camera pose estimation and a 27% improvement in point cloud reconstruction F-Score, significantly outperforming state-of-the-art methods—especially in challenging regions with weak texture or low illumination.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent advances in data-driven geometric multi-view 3D reconstruction foundation models (e.g., DUSt3R) have shown remarkable performance across various 3D vision tasks, facilitated by the release of large-scale, high-quality 3D datasets. However, as we observed, constrained by their matching-based principles, the reconstruction quality of existing models suffers significant degradation in challenging regions with limited matching cues, particularly in weakly textured areas and low-light conditions. To mitigate these limitations, we propose to harness the inherent robustness of monocular geometry estimation to compensate for the inherent shortcomings of matching-based methods. Specifically, we introduce a monocular-guided refinement module that integrates monocular geometric priors into multi-view reconstruction frameworks. This integration substantially enhances the robustness of multi-view reconstruction systems, leading to high-quality feed-forward reconstructions. Comprehensive experiments across multiple benchmarks demonstrate that our method achieves substantial improvements in both mutli-view camera pose estimation and point cloud accuracy.

Problem

• Improves 3D reconstruction in weakly textured areas
• Enhances multi-view systems with monocular geometric priors
• Addresses degradation in low-light and matching-limited regions

Innovation

• Monocular-guided refinement module integration
• Enhances robustness with monocular geometric priors
• Improves multi-view camera pose estimation accuracy