Gaussian Splatting with Discretized SDF for Relightable Assets

2025-07-21

AI Summary

To address the challenge of imposing geometric constraints in inverse rendering with 3D Gaussian Splatting (3DGS), this paper proposes a relightable asset generation method embedding a discretized Signed Distance Field (SDF). The core innovation lies in quantizing the SDF and encoding it per Gaussian primitive, coupled with an SDF-to-opacity mapping that enables efficient, ray-marching-free rendering. Additionally, a projection consistency loss is introduced to enforce the projection of Gaussian centers onto the SDF zero-level set, thereby implicitly encoding geometry without explicit mesh or voxel representations. This approach preserves the memory efficiency of 3DGS while significantly improving relighting fidelity and geometric consistency. Experiments demonstrate superior performance over state-of-the-art Gaussian-based inverse rendering methods across multiple benchmarks.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

3D Gaussian splatting (3DGS) has shown its detailed expressive ability and highly efficient rendering speed in the novel view synthesis (NVS) task. The application to inverse rendering still faces several challenges, as the discrete nature of Gaussian primitives makes it difficult to apply geometry constraints. Recent works introduce the signed distance field (SDF) as an extra continuous representation to regularize the geometry defined by Gaussian primitives. It improves the decomposition quality, at the cost of increasing memory usage and complicating training. Unlike these works, we introduce a discretized SDF to represent the continuous SDF in a discrete manner by encoding it within each Gaussian using a sampled value. This approach allows us to link the SDF with the Gaussian opacity through an SDF-to-opacity transformation, enabling rendering the SDF via splatting and avoiding the computational cost of ray marching.The key challenge is to regularize the discrete samples to be consistent with the underlying SDF, as the discrete representation can hardly apply the gradient-based constraints (eg Eikonal loss). For this, we project Gaussians onto the zero-level set of SDF and enforce alignment with the surface from splatting, namely a projection-based consistency loss. Thanks to the discretized SDF, our method achieves higher relighting quality, while requiring no extra memory beyond GS and avoiding complex manually designed optimization. The experiments reveal that our method outperforms existing Gaussian-based inverse rendering methods. Our code is available at https://github.com/NK-CS-ZZL/DiscretizedSDF.

Problem

• Improving relighting quality with discretized SDF
• Reducing memory usage in Gaussian-based inverse rendering
• Avoiding complex optimization in SDF regularization

Innovation

• Discretized SDF encoded within each Gaussian
• SDF-to-opacity transformation for splatting rendering
• Projection-based consistency loss for SDF regularization