G3Splat: Geometrically Consistent Generalizable Gaussian Splatting

2025-12-19

AI Summary

3D Gaussian Splatting suffers from geometric inconsistency and lack of geometric interpretability under pose-free supervision. Method: This work introduces the first self-supervised framework that explicitly models geometric consistency by decoupling view synthesis from geometry reconstruction. It incorporates triple geometric priors—depth, surface normals, and relative camera poses—optimized jointly via a multi-view structure prediction network (pretrained on RE10K) embedded within a differentiable Gaussian rendering pipeline. Contribution/Results: The approach enables end-to-end learning of geometrically faithful Gaussian scene representations with zero-shot cross-dataset generalization. Experiments establish new state-of-the-art performance on RE10K for novel-view synthesis, geometric reconstruction, and relative pose estimation. When transferred to ScanNet, it reduces geometric error by 37% and improves pose estimation accuracy by 2.1× over prior methods.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Unsupervised & Self-Supervised Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

3D Gaussians have recently emerged as an effective scene representation for real-time splatting and accurate novel-view synthesis, motivating several works to adapt multi-view structure prediction networks to regress per-pixel 3D Gaussians from images. However, most prior work extends these networks to predict additional Gaussian parameters -- orientation, scale, opacity, and appearance -- while relying almost exclusively on view-synthesis supervision. We show that a view-synthesis loss alone is insufficient to recover geometrically meaningful splats in this setting. We analyze and address the ambiguities of learning 3D Gaussian splats under self-supervision for pose-free generalizable splatting, and introduce G3Splat, which enforces geometric priors to obtain geometrically consistent 3D scene representations. Trained on RE10K, our approach achieves state-of-the-art performance in (i) geometrically consistent reconstruction, (ii) relative pose estimation, and (iii) novel-view synthesis. We further demonstrate strong zero-shot generalization on ScanNet, substantially outperforming prior work in both geometry recovery and relative pose estimation. Code and pretrained models are released on our project page (https://m80hz.github.io/g3splat/).

Problem

• Addresses geometric ambiguity in self-supervised 3D Gaussian splatting
• Enforces geometric priors for consistent 3D scene representations
• Improves reconstruction, pose estimation, and novel-view synthesis performance

Innovation

• Enforces geometric priors for consistent 3D scene representation
• Uses self-supervision for pose-free generalizable Gaussian splatting
• Achieves state-of-the-art in geometry, pose, and view synthesis