Gaussian Sculpting: End-to-End Controllable Surface Reconstruction via Field Optimization

2026-08-11

AI Summary

This work addresses the challenge of inaccurate geometry recovery in 3D Gaussian Splatting under limited viewing conditions, where irregular Gaussian primitives hinder effective geometric refinement. To overcome this, the authors propose an end-to-end differentiable framework that anchors Gaussian primitives to a differentiable signed distance field (SDF). The approach employs a bilevel optimization strategy: the outer loop updates the underlying geometry via the SDF, while the inner loop refines Gaussian attributes. Additionally, a Gaussian-surface consistency constraint and an octree-based multi-resolution subdivision mechanism are introduced to suppress redundant surfaces and complete missing structures. This method achieves high-quality joint reconstruction of geometry and appearance even from low-resolution inputs.

Abstract

3D Gaussian Splatting (3DGS) has recently enabled real-time novel view synthesis with impressive quality. However, it struggles to recover accurate surfaces under limited viewpoints and due to the inherent irregularity of Gaussian primitives. The resulting geometric errors are notoriously difficult to correct manually. To address these issues, we propose Gaussian Sculpting, a fully differentiable end-to-end framework for high-quality surface reconstruction. Our key insight is to anchor Gaussians onto an evolving differentiable surface, allowing them to guide signed distance field (SDF) optimization instead of extracting the surface only during post-processing. To enable stable gradient isolation during joint optimization, we design a bi-level training strategy in which the outer loop optimizes the geometry represented by the SDF, while the inner loop updates the Gaussians with the geometry fixed. We further impose constraints on Gaussian parameters to ensure consistency with the underlying surface, thereby improving both geometric and appearance fidelity during optimization. In addition, we introduce a multi-resolution subdivision scheme based on octree-like partitioning to preserve fine details while reducing memory consumption. Experiments on object-level scenes demonstrate that our method effectively removes redundant surfaces, recovers missing structures caused by limited viewpoints, and achieves strong reconstruction quality even at relatively low resolutions.

Problem

• 3D Gaussian Splatting
• surface reconstruction
• limited viewpoints
• geometric errors
• irregular primitives

Innovation

• Gaussian Splatting
• Signed Distance Field (SDF)
• Differentiable Rendering
• Bi-level Optimization
• Surface Reconstruction