VI3: Grounding Pretrained 3D Foundation Models with Inertial Cues

2026-09-03

AI Summary

本文提出VI3框架，通过IMU读数为预训练的3D基础模型提供度量锚定，解决其绝对尺度预测不准确的问题。

Abstract

3D foundation models (3DFMs) excel at predicting camera poses and dense depth from multiple views of a scene, showcasing strong zero-shot generalization. However, as metric scale is not observable from monocular images, their absolute scale predictions are typically inaccurate. Inertial measurement units (IMUs), present in most devices, naturally complement monocular cameras by observing scaled motion. We introduce VI3, a model-agnostic framework that metrically anchors a pretrained 3DFM using only IMU readings. VI3 initializes and preintegrates the IMU to obtain a metric motion reference, which is then used to recover the scale of the 3DFM outputs. Our method includes adaptable anchoring strategies tailored to diverse 3DFM architectures. Experiments on synthetic and real aerial datasets demonstrate that VI3 recovers metric scale without ground-truth supervision while preserving geometric consistency, acting as a fine refinement under well-conditioned motion and as a strong prior when motion is less informative.

Problem

• 3D foundation models
• metric scale
• monocular images
• inertial measurement units

Innovation

• Inertial Measurement Units
• Metric Scale Recovery
• Model-Agnostic Framework
• 3D Foundation Models