Gbake: Baking 3D Gaussian Splats into Reflection Probes

2025-07-02

AI Summary

To address reflection distortions and lighting inconsistencies in hybrid 3D scenes—arising from the intrinsic coupling of geometry and appearance in conventional mesh models when integrated with 3D Gaussian Splatting—this paper proposes the first reflection probe baking method specifically designed for 3D Gaussian point clouds. Our approach integrates ray sampling and environment radiance field reconstruction into the Gaussian rendering pipeline, explicitly encoding the implicit Gaussian scene into Unity-compatible cubic reflection probes. This enables physically consistent relighting of mesh objects. Experiments demonstrate that our method significantly enhances reflection fidelity and visual coherence of mesh assets within Gaussian–mesh hybrid scenes. Crucially, it achieves high-quality cross-representation (Gaussian–Mesh) lighting fusion without compromising the computational efficiency inherent to Gaussian splatting.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Mixed Discrete/Continuous Search
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Security and Privacy: Data transparency and provenance
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

The growing popularity of 3D Gaussian Splatting has created the need to integrate traditional computer graphics techniques and assets in splatted environments. Since 3D Gaussian primitives encode lighting and geometry jointly as appearance, meshes are relit improperly when inserted directly in a mixture of 3D Gaussians and thus appear noticeably out of place. We introduce GBake, a specialized tool for baking reflection probes from Gaussian-splatted scenes that enables realistic reflection mapping of traditional 3D meshes in the Unity game engine.

Problem

• Integrate traditional graphics with 3D Gaussian Splatting
• Fix improper lighting of meshes in Gaussian environments
• Bake reflection probes for realistic mesh rendering in Unity

Innovation

• Baking reflection probes from Gaussian splats
• Enabling realistic mesh relighting in Unity
• Integrating traditional graphics with Gaussian splatting