ORBIT++: Benchmarking SfM in the Wild with 360° Video

2026-08-22

AI Summary

为了解决复杂视频中相机姿态估计不准确的问题，研究者通过360°全景视频构建了新的基准ORBIT，提供了可靠的真值以评估SfM方法。

Abstract

Structure-from-Motion (SfM) is a cornerstone of 3D perception, yet current methods often fail when applied to complex videos involving challenging camera motions or dynamic scenes. Compounding the problem, the field lacks reliable ground-truth benchmarks for such difficult scenarios, making it hard to gauge real-world progress or to pinpoint where improvements are most needed. To address this gap, we introduce a new benchmark for evaluating camera pose estimation. Our key insight is to leverage online panoramic 360° video as a source of data from which to construct challenging clips, while still enabling robust ground-truth trajectory recovery. The panoramic nature of these videos provides richer visual context for tracking camera motion, even when parts of the view are affected by blur, motion, or dynamic objects. After tracking camera motion across full 360° videos, we crop and reproject selected portions to generate perspective-view clips that serve as our benchmark, called ORBIT. Experiments show that COLMAP, as well as recent optimization-based and feed-forward SfM methods struggle to accurately estimate camera poses on our benchmark. Hence, ORBIT provides a valuable testbed where researchers can meaningfully measure progress on truly challenging, real-world SfM problems.

Problem

• Structure-from-Motion
• camera pose estimation
• benchmarking
• dynamic scenes
• 360 video

Innovation

• Structure-from-Motion
• panoramic 360° video
• camera pose estimation
• ground-truth trajectory recovery