Editable Physically-based Reflections in Raytraced Gaussian Radiance Fields

2026-06-29

AI Summary

While existing 3D Gaussian splatting methods can simulate specular reflections, they rely on non-physical “faked” geometry, making it difficult to consistently edit both reflectors and reflected objects. This work proposes a physically based, editable multi-bounce specular reflection method that learns to decompose input views into diffuse and specular components, jointly optimizes the underlying diffuse scene, and integrates path tracing with an efficient ray-tracing scheme tailored for 3D Gaussian primitives. For the first time in Gaussian radiance fields, this approach enables the reconstruction of physically accurate reflective structures. It supports real-time, physically consistent interactions—including roughness adjustment and multi-bounce editing—and demonstrates high-quality results on synthetic data while validating its feasibility on real-world scenes.

Abstract

Radiance fields such as 3D Gaussian Splatting allow real-time rendering of scenes captured from photos. They also reconstruct most specular reflections with high visual quality, but typically model them with "fake" reflected geometry, using primitives behind the reflector. Our goal is to correctly reconstruct the reflector and the reflected objects such as to make specular reflections editable. We present a proof of concept which exploits promising learning-based methods to extract diffuse and specular buffers from photos, as well as geometry and BRDF buffers. Our method builds on three key components. First, by using diffuse and specular buffers of input training views, we optimize a diffuse version of the scene and use path tracing to efficiently generate physically based specular reflections. Second, we present a specialized training method that allows this process to converge. Finally, we present a fast ray tracing algorithm for 3D Gaussian primitives that enables efficient multi-bounce reflections. Our method reconstructs reflectors and reflected objects, including those not seen in the input images, in a unique scene representation. Our solution allows real-time, consistent editing of captured scenes with specular reflections, including multi-bounce effects, changing roughness, and more. We mainly show results using ground truth buffers from synthetic scenes, and also preliminary results in real scenes with currently imperfect learning-based buffers. Code and data are available at: https://repo-sam.inria.fr/nerphys/editable-gaussian-reflections/

Problem

• specular reflections
• editable rendering
• radiance fields
• 3D Gaussian Splatting
• physically-based rendering

Innovation

• Editable Reflections
• Physically-based Rendering
• Gaussian Radiance Fields
• Ray Tracing
• Specular-Diffuse Decomposition