F-RNG: Feed-Forward Relightable Neural Gaussians

2026-05-25

AI Summary

Existing methods struggle to efficiently generate relightable 3D Gaussian assets from sparse views, often relying on dense inputs or exhibiting limited generalization. This work proposes F-RNG, the first framework to enable feedforward relightable 3D Gaussian generation without retraining large reconstruction models. F-RNG leverages priors from a Large Reconstruction Model (LRM) and an Intrinsic Decomposition Model (IDM), synthesizing geometry through latent space interpolation and constructing a general-purpose neural renderer via prior-guided appearance distillation. Compared to state-of-the-art approaches, F-RNG achieves approximately 25× faster relighting, improves image quality by about 2.0 dB, and substantially reduces training resource requirements, thereby supporting plug-and-play deployment with future models.

Abstract

Capturing relightable 3D assets from real-world objects is a widely researched problem. Several per-scene optimization-based methods, based on 3D Gaussian splatting (3DGS), support relighting; however, they usually require dense input views, and their overfitting nature makes it difficult to generalize across scenes. Unlike per-scene optimization methods, generalized feed-forward models can directly reconstruct Gaussians from sparse input views. However, the resulting assets have baked-in illumination and cannot be easily used for relighting. In this paper, we present F-RNG, a feed-forward framework that directly generates relightable 3DGS assets from sparse-view inputs. Training such a model from scratch can require massive data and computing resources, and it is especially challenging to generate relightable assets in a feed-forward manner with acceptable cost. We develop F-RNG upon an existing large reconstruction model (LRM) to extract relightable representations, while also utilizing priors from an intrinsic decomposition model (IDM). Specifically, we first introduce a latent-interpolated fine-grained geometry synthesis to enhance the LRM's geometry representation. Second, we propose a prior-guided relightable appearance distillation to extract relightable neural representations by incorporating IDM priors. Finally, a universal neural renderer enables flexible and high-fidelity relighting. F-RNG requires neither re-training nor fine-tuning of the underlying LRMs, thus can automatically benefit from better LRMs and IDMs in the future. With only small networks that can be trained with affordable data and computational resources, F-RNG avoids the repetitive inference of large models under different light conditions. By comparison to the state-of-the-art LRM-based relighting method, F-RNG achieves ~25x faster relighting, as well as superior quality (~+2.0 dB).

Problem

• relightable 3D reconstruction
• sparse-view input
• 3D Gaussian splatting
• feed-forward model
• neural rendering

Innovation

• Feed-Forward Relighting
• Relightable Neural Gaussians
• 3D Gaussian Splatting
• Intrinsic Decomposition Prior
• Latent-Interpolated Geometry