RT-GS: Gaussian Splatting with Reflection and Transmittance Primitives

2026-04-01

AI Summary

Traditional Gaussian splatting struggles to simultaneously model specular reflections and the appearance of surfaces behind translucent materials, limiting the photorealism of novel view synthesis. This work proposes RT-GS, a unified framework that introduces physically driven reflective and transmissive Gaussian primitives into Gaussian splatting for the first time. By integrating a microsurface-based material model with differentiable ray tracing, RT-GS jointly optimizes the geometry and appearance of both distant reflections and objects seen through transparent media. The method significantly enhances novel view synthesis quality in complex scenes featuring strong specular and transmission effects, successfully reconstructing structures behind transparent objects and generating realistic reflections.

Abstract

Gaussian Splatting is a powerful tool for reconstructing diffuse scenes, but it struggles to simultaneously model specular reflections and the appearance of objects behind semi-transparent surfaces. These specular reflections and transmittance are essential for realistic novel view synthesis, and existing methods do not properly incorporate the underlying physical processes to simulate them. To address this issue, we propose RT-GS, a unified framework that integrates a microfacet material model and ray tracing to jointly model specular reflection and transmittance in Gaussian Splatting. We accomplish this by using separate Gaussian primitives for reflections and transmittance, which allow modeling distant reflections and reconstructing objects behind transparent surfaces concurrently. We utilize a differentiable ray tracing framework to obtain the specular reflection and transmittance appearance. Our experiments demonstrate that our method successfully produces reflections and recovers objects behind transparent surfaces in complex environments, achieving significant qualitative improvements over prior methods where these specular light interactions are prominent.

Problem

• Gaussian Splatting
• specular reflection
• transmittance
• novel view synthesis
• transparent surfaces

Innovation

• Gaussian Splatting
• specular reflection
• transmittance
• differentiable ray tracing
• microfacet model