IVGT: Implicit Visual Geometry Transformer for Neural Scene Representation

2026-05-15

AI Summary

This work proposes an Implicit Visual Geometry Transformer for reconstructing consistent and continuous 3D geometry and appearance from pose-free multi-view images. The method learns an implicit neural scene representation in a canonical coordinate system, enabling spatial queries at arbitrary 3D locations through a lightweight decoder that jointly predicts signed distance functions and color. By eschewing explicit point-cloud regression in favor of implicit continuous geometric modeling, the approach integrates 2D supervision with 3D geometric regularization to achieve unified optimization across multiple datasets. The framework demonstrates state-of-the-art performance in mesh and point cloud reconstruction, novel view synthesis, depth and normal estimation, and camera pose recovery, exhibiting strong cross-scene generalization capabilities.

Abstract

Reconstructing coherent 3D geometry and appearance from unposed multi-view images is a fundamental yet challenging problem in computer vision. Most existing visual geometry foundation models predict explicit geometry by regressing pixel-aligned pointmaps, often suffering from redundancy and limited geometric continuity. We propose IVGT, an Implicit Visual Geometry Transformer that implicitly models continuous and coherent geometry from pose-free multi-view images. This formulation learns a continuous neural scene representation in a canonical coordinate system and supports continuous spatial queries at any 3D positions, retrieving local features to predict signed distance (SDF) values and colors using lightweight decoders. It allows direct extraction of continuous and coherent surface geometry, enabling rendering of RGB images, depth maps, and surface normal maps from arbitrary viewpoints. We train IVGT via multi-dataset joint optimization with 2D supervision and 3D geometric regularization. IVGT demonstrates generalization across scenes and achieves strong performance on various tasks, including mesh and point cloud reconstruction, novel view synthesis, depth and surface normal estimation, and camera pose estimation.

Problem

• neural scene representation
• 3D reconstruction
• multi-view images
• implicit geometry
• pose-free

Innovation

• Implicit Neural Representation
• Visual Geometry Transformer
• Pose-Free Multi-View Reconstruction
• Signed Distance Function (SDF)
• Neural Scene Representation