MGSfM: Multi-Camera Geometry Driven Global Structure-from-Motion

2025-07-04

AI Summary

To address the insufficient robustness and low efficiency of global Structure-from-Motion (SfM) in multi-camera systems, this paper proposes a novel global motion averaging framework. Methodologically, it decouples rotation and translation estimation: hierarchical relative rotation averaging improves rotational consistency, while a convex translation optimization objective jointly incorporates camera–camera and camera–point constraints; additionally, distance-based initialization and an unbiased non-bilinear angular loss function are introduced. Our key contribution is the first integration of intrinsic, fixed inter-camera pose constraints—encoded as hard geometric priors—into global motion averaging, significantly enhancing robustness against outliers and noise. Experiments demonstrate that the method achieves accuracy comparable to or exceeding incremental SfM on large-scale datasets, while substantially outperforming existing global SfM approaches in computational efficiency. The framework thus offers strong practical value for real-world multi-camera 3D reconstruction deployment.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Multi-camera systems are increasingly vital in the environmental perception of autonomous vehicles and robotics. Their physical configuration offers inherent fixed relative pose constraints that benefit Structure-from-Motion (SfM). However, traditional global SfM systems struggle with robustness due to their optimization framework. We propose a novel global motion averaging framework for multi-camera systems, featuring two core components: a decoupled rotation averaging module and a hybrid translation averaging module. Our rotation averaging employs a hierarchical strategy by first estimating relative rotations within rigid camera units and then computing global rigid unit rotations. To enhance the robustness of translation averaging, we incorporate both camera-to-camera and camera-to-point constraints to initialize camera positions and 3D points with a convex distance-based objective function and refine them with an unbiased non-bilinear angle-based objective function. Experiments on large-scale datasets show that our system matches or exceeds incremental SfM accuracy while significantly improving efficiency. Our framework outperforms existing global SfM methods, establishing itself as a robust solution for real-world multi-camera SfM applications. The code is available at https://github.com/3dv-casia/MGSfM/.

Problem

• Enhances robustness in multi-camera Structure-from-Motion systems
• Improves global motion averaging for multi-camera configurations
• Optimizes rotation and translation averaging for better accuracy

Innovation

• Hierarchical rotation averaging for multi-camera systems
• Hybrid translation averaging with dual constraints
• Convex and angle-based objective functions refinement