Glob3R: Global Structure-from-Motion with 3D Foundation Models

2026-07-10

AI Summary

Existing 3D foundation models suffer from insufficient accuracy when reconstructing large-scale unordered image collections or long sequences, and patch-based processing often leads to drift and inconsistency. This work proposes a novel approach that introduces a lightweight dense matching head on top of a frozen Pi3X backbone to predict deformations between a reference frame and its neighboring views, thereby establishing reliable multi-view feature trajectories. Furthermore, a keyframe sliding window mechanism is designed to integrate the prior knowledge from the 3D foundation model with global geometric optimization, enabling scalable global motion averaging and bundle adjustment. The method significantly outperforms current feed-forward foundation models and scalable reconstruction techniques across indoor, outdoor, large-scale driving, and unordered structure-from-motion benchmarks, achieving notable improvements in reconstruction accuracy, robustness, and neural rendering quality.

Abstract

Recent 3D geometric foundation models, such as VGGT, provide robust feed-forward 3D reconstruction by directly predicting camera poses and 3D scene points from input images. However, their results remain inaccurate, and scaling them to long sequences or large unordered image sets typically requires chunk-wise processing, which can introduce drift and inconsistency. We present Glob3R, a global SfM-style reconstruction built on 3D foundation models. Our key idea is to explicitly optimize feed-forward geometric predictions. To this end, we augment a frozen Pi3X backbone with a lightweight dense matching head that predicts image warps between selected reference frames and neighboring views. These dense warps are converted into sparse but reliable multi-view feature tracks, which provide correspondence constraints for global optimization. We further introduce a keyframe-based sliding-window association strategy that propagates tracks and relative poses across overlapping windows, enabling scalable reconstruction. Finally, we perform global motion averaging and bundle adjustment to refine camera poses, reduce scale inconsistencies, and recover dense scene geometry. Extensive experiments on indoor, outdoor, large-scale driving, and unordered SfM benchmarks demonstrate that Glob3R achieves robust and accurate reconstruction. It consistently improves over feed-forward foundation-model baselines and recent scalable reconstruction methods, while being more robust than classical SfM pipelines. The refined poses also lead to higher-quality neural rendering, validating the benefit of combining foundation-model priors with global geometric optimization. Project page: https://junyuandeng.github.io/Glob3r

Problem

• 3D reconstruction
• Structure-from-Motion
• foundation models
• scale inconsistency
• drift

Innovation

• 3D foundation models
• global Structure-from-Motion
• dense matching
• sliding-window optimization
• bundle adjustment