MAGiSt3R: Multi-Agent Feed-forward 3D Reconstruction from Monocular RGB Videos

2026-07-16

AI Summary

This work addresses the challenge of cumulative drift in monocular RGB video-based 3D scene reconstruction and camera pose estimation by introducing a multi-agent feedforward reconstruction framework. It is the first to integrate a multi-agent mechanism into feedforward monocular 3D reconstruction, leveraging local point map regression and multi-level fusion to produce globally consistent point clouds while incorporating pose graph optimization to effectively suppress trajectory drift. Built upon the 3R family of feedforward models and a novel multi-agent fusion module named MAGMA, the proposed method significantly outperforms existing approaches on both synthetic and real-world datasets, achieving state-of-the-art performance in reconstruction accuracy, pose fidelity, and robustness, with an inference speed approaching 10 FPS.

Abstract

This paper presents MAGiSt3R, a multi-agent 3D reconstruction framework performing reconstruction and camera tracking for monocular RGB videos at almost 10 FPS. MAGiSt3R relies on a feed-forward model from the 3R family to process RGB videos and regress local point maps, and on a merging model, MAGMA, that combines local maps at both intra-agent and inter-agent levels to obtain the final global point map. Furthermore, MAGiSt3R performs pose graph optimization to mitigate cumulative camera drift occurring along the feed-forward pipeline. We evaluate MAGiSt3R on both synthetic and real-world datasets, demonstrating its superior reconstruction and camera tracking accuracy compared to state-of-the-art approaches.

Problem

• 3D reconstruction
• monocular RGB videos
• camera tracking
• camera drift
• multi-agent

Innovation

• multi-agent
• feed-forward 3D reconstruction
• monocular RGB video
• pose graph optimization
• MAGMA