GS-2DGS: Geometrically Supervised 2DGS for Reflective Object Reconstruction

2025-06-16

AI Summary

Reconstructing highly reflective objects in 3D suffers from severe geometric distortion and surface noise due to strong view-dependent appearance. To address this, we propose the first real-time reconstruction framework that integrates 2D Gaussian Splatting (2DGS) with geometric priors from vision foundation models. Specifically, we leverage SAM and DINO to extract implicit geometric constraints, and introduce differentiable geometric distillation alongside multi-view consistency regularization—enabling, for the first time, explicit geometric supervision within 2DGS. Our method effectively suppresses surface noise while preserving structural coherence. Quantitative and qualitative evaluations demonstrate reconstruction accuracy on par with state-of-the-art signed distance function (SDF)-based methods on both synthetic and real-world scenes, superior relighting quality, and over 10× faster inference. The framework achieves a unique balance among fine-detail fidelity, geometric robustness, and real-time performance.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Feature Construction/Reformulation

Application Category

• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

3D modeling of highly reflective objects remains challenging due to strong view-dependent appearances. While previous SDF-based methods can recover high-quality meshes, they are often time-consuming and tend to produce over-smoothed surfaces. In contrast, 3D Gaussian Splatting (3DGS) offers the advantage of high speed and detailed real-time rendering, but extracting surfaces from the Gaussians can be noisy due to the lack of geometric constraints. To bridge the gap between these approaches, we propose a novel reconstruction method called GS-2DGS for reflective objects based on 2D Gaussian Splatting (2DGS). Our approach combines the rapid rendering capabilities of Gaussian Splatting with additional geometric information from foundation models. Experimental results on synthetic and real datasets demonstrate that our method significantly outperforms Gaussian-based techniques in terms of reconstruction and relighting and achieves performance comparable to SDF-based methods while being an order of magnitude faster. Code is available at https://github.com/hirotong/GS2DGS

Problem

• Reconstructing reflective objects with view-dependent appearances
• Overcoming over-smoothed surfaces in SDF-based methods
• Reducing noise in Gaussian Splatting surface extraction

Innovation

• Uses 2D Gaussian Splatting for reflective objects
• Integrates geometric data from foundation models
• Achieves fast high-quality reconstruction and relighting