3DGSR: Implicit Surface Reconstruction with 3D Gaussian Splatting

2024-03-30

ACM Transactions on Graphics

AI Summary

Existing 3D reconstruction methods struggle to accurately recover occluded regions and complex geometries under sparse-view settings, often yielding surface distortions, missing fine details, or spurious structures. To address this, we propose 3DGSR—a novel framework that pioneers the integration of implicit Signed Distance Fields (SDFs) into the 3D Gaussian Splatting (3DGS) pipeline. Our method establishes a coupled alignment and joint optimization mechanism between SDFs and 3D Gaussians. Furthermore, we introduce depth- and normal-guided volumetric rendering constraints to enforce geometric consistency, thereby mitigating supervision scarcity and enabling bidirectional enhancement: SDFs benefit from 3DGS’s high-fidelity rendering capability, while 3DGS gains improved geometric fidelity through continuous SDF modeling. 3DGSR preserves the efficiency and visual quality of 3DGS while significantly improving surface reconstruction accuracy and detail preservation. It converges faster and achieves state-of-the-art performance on standard benchmarks.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

In this paper, we present an implicit surface reconstruction method with 3D Gaussian Splatting (3DGS), namely 3DGSR, that allows for accurate 3D reconstruction with intricate details while inheriting the high efficiency and rendering quality of 3DGS. The key insight is to incorporate an implicit signed distance field (SDF) within 3D Gaussians for surface modeling, and to enable the alignment and joint optimization of both SDF and 3D Gaussians. To achieve this, we design coupling strategies that align and associate the SDF with 3D Gaussians, allowing for unified optimization and enforcing surface constraints on the 3D Gaussians. With alignment, optimizing the 3D Gaussians provides supervisory signals for SDF learning, enabling the reconstruction of intricate details. However, this only offers sparse supervisory signals to the SDF at locations occupied by Gaussians, which is insufficient for learning a continuous SDF. Then, to address this limitation, we incorporate volumetric rendering and align the rendered geometric attributes (depth, normal) with that derived from 3DGS. In sum, these two designs allow SDF and 3DGS to be aligned, jointly optimized, and mutually boosted. Our extensive experimental results demonstrate that our 3DGSR enables high-quality 3D surface reconstruction while preserving the efficiency and rendering quality of 3DGS. Besides, our method competes favorably with leading surface reconstruction techniques while offering a more efficient learning process and much better rendering qualities.

Problem

• 3D Reconstruction
• Hidden or Complex Surfaces
• Accuracy and Detail

Innovation

• 3DGSR
• Gaussian Splatting
• Signed Distance Fields