ColabSfM: Collaborative Structure-from-Motion by Point Cloud Registration

2025-03-21

AI Summary

Existing collaborative Structure-from-Motion (ColabSfM) approaches are hindered by the lack of scalable distributed reconstruction registration methods and task-specific training data. To address this, we formally define and establish the SfM point cloud registration task. We propose a synthetic data generation pipeline that simulates partial reconstructions using real-world camera trajectories, explicitly modeling geometric structure and realistic noise degradations. Furthermore, we design RefineRoITr—a lightweight neural refinement module—integrated into the RoITr architecture to enhance robustness against geometric inconsistencies and noise. Evaluated on our newly constructed SfM registration benchmark, our method significantly outperforms existing registration baselines. It is the first to enable scalable, robust, multi-device collaborative alignment and sharing of SfM maps. The code and dataset are publicly released.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Structure-from-Motion (SfM) is the task of estimating 3D structure and camera poses from images. We define Collaborative SfM (ColabSfM) as sharing distributed SfM reconstructions. Sharing maps requires estimating a joint reference frame, which is typically referred to as registration. However, there is a lack of scalable methods and training datasets for registering SfM reconstructions. In this paper, we tackle this challenge by proposing the scalable task of point cloud registration for SfM reconstructions. We find that current registration methods cannot register SfM point clouds when trained on existing datasets. To this end, we propose a SfM registration dataset generation pipeline, leveraging partial reconstructions from synthetically generated camera trajectories for each scene. Finally, we propose a simple but impactful neural refiner on top of the SotA registration method RoITr that yields significant improvements, which we call RefineRoITr. Our extensive experimental evaluation shows that our proposed pipeline and model enables ColabSfM. Code is available at https://github.com/EricssonResearch/ColabSfM

Problem

• Lack scalable methods for SfM reconstruction registration
• Current datasets fail to train effective registration models
• Need joint reference frame estimation for collaborative SfM

Innovation

• Point cloud registration for SfM reconstructions
• SfM registration dataset generation pipeline
• Neural refiner RefineRoITr for registration