Mesh2GS: White-Box 3DGS Construction via Plenoptic Sampling

2026-06-20

AI Summary

This work addresses the challenge of efficiently constructing a 3D Gaussian Splatting (3DGS) representation from mesh models that supports global illumination and non-Lambertian reflectance. Building upon plenoptic function sampling theory, the authors propose a white-box construction framework that, for the first time, derives the minimal view sampling rate required for 3DGS and directly generates Gaussians from input meshes. The approach incorporates an albedo-shading decomposition mechanism and a neural lighting enhancement module, enabling Nyquist-rate sampling quality and accurate modeling of non-Lambertian effects such as specular highlights. The resulting method achieves superior rendering fidelity compared to existing techniques while retaining real-time rendering performance, offering both practical utility and scalability.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a promising method for high-quality, real-time 3D reconstruction. To associate 3DGS with mesh representations, existing methods primarily focus on 3DGS-to-mesh reconstruction from multi-view images. In contrast, the problem of converting a mesh into 3DGS has received comparatively less attention. Instead of relying on heuristic strategies that bind 3D Gaussians to the mesh, we propose a novel white-box 3DGS construction framework, termed Mesh2GS, which generates 3DGS directly from mesh geometry based on plenoptic sampling theory, achieving Nyquist-level performance for high-quality global illumination rendering. Firstly, we propose a plenoptic sampling guided 3DGS construction strategy that theoretically derives the minimum sampling rate of the sampled views and the distribution of 3D Gaussians. Second, we propose a novel 3DGS update procedure with albedo--shading decomposition for efficient global-illumination capture. Finally, we introduce a neural illumination enhancement module to handle non-Lambertian effects. Experimental results demonstrate that our method surpasses state-of-the-art baselines and is practically effective for both real-time shared rendering and non-Lambertian effects capturing specular highlights. The project code will be released upon acceptance.

Problem

• 3D Gaussian Splatting
• mesh-to-3DGS conversion
• plenoptic sampling
• global illumination
• non-Lambertian effects

Innovation

• 3D Gaussian Splatting
• Plenoptic Sampling
• White-box Construction
• Global Illumination
• Non-Lambertian Rendering