SpectralSplat: Appearance-Disentangled Feed-Forward Gaussian Splatting for Driving Scenes

2026-04-03

AI Summary

This work addresses the limitation of existing feed-forward 3D Gaussian splatting methods, which entangle geometry with transient appearance factors such as lighting and weather, thereby hindering relighting and appearance transfer. To overcome this, we propose SpectralSplat—the first approach within a feed-forward Gaussian splatting framework to explicitly disentangle geometry from appearance. Our method employs a shared MLP to generate an appearance-invariant base color field and introduces an adaptive field modulated by a global appearance embedding. Leveraging physics-guided intrinsic decomposition and diffusion models, we synthesize paired training data and adopt a hybrid relighting strategy for joint optimization. SpectralSplat achieves temporally consistent re-rendering across time and environmental conditions while enabling controllable appearance editing, all without compromising reconstruction fidelity.

Abstract

Feed-forward 3D Gaussian Splatting methods have achieved impressive reconstruction quality for autonomous driving scenes, yet they entangle scene geometry with transient appearance properties such as lighting, weather, and time of day. This coupling prevents relighting, appearance transfer, and consistent rendering across multi-traversal data captured under varying environmental conditions. We present SpectralSplat, a method that disentangles appearance from geometry within a feed-forward Gaussian Splatting framework. Our key insight is to factor color prediction into an appearance-agnostic base stream and and appearance-conditioned adapted stream, both produced by a shared MLP conditioned on a global appearance embedding derived from DINOv2 features. To enforce disentanglement, we train with paired observations generated by a hybrid relighting pipeline that combines physics-based intrinsic decomposition with diffusion based generative refinement, and supervise with complementary consistency, reconstruction, cross-appearance, and base color losses. We further introduce an appearance-adaptable temporal history that stores appearance-agnostic features, enabling accumulated Gaussians to be re-rendered under arbitrary target appearances. Experiments demonstrate that SpectralSplat preserves the reconstruction quality of the underlying backbone while enabling controllable appearance transfer and temporally consistent relighting across driving sequences.

Problem

• Gaussian Splatting
• appearance disentanglement
• autonomous driving scenes
• relighting
• appearance transfer

Innovation

• appearance disentanglement
• Gaussian Splatting
• relighting
• feed-forward 3D reconstruction
• autonomous driving scenes