InvSplat: Inverse Feed-Forward Scene Splatting

2026-07-02

AI Summary

Existing inverse rendering methods suffer from limitations in multi-view consistency, explicit 3D representation, and computational efficiency. This work proposes a feed-forward multi-view inverse rendering framework that, for the first time, embeds decoupled and physically plausible material attributes—such as albedo, metallicity, and roughness—into a structured 3D Gaussian representation, enabling joint prediction of geometry and reflectance parameters in a single forward pass. By integrating a multi-view reconstruction backbone with priors for material estimation, the method achieves highly consistent novel-view synthesis and accurate material recovery on both synthetic and real-world datasets, while supporting physically based relighting and modeling of view-dependent effects.

Abstract

Inverse rendering aims to recover both 3D geometry and physically meaningful material properties from images, enabling applications such as relighting and novel view synthesis. Optimization-based methods achieve high fidelity but require costly per-scene fitting, while image-space learning-based approaches often suffer from multi-view inconsistencies and lack an explicit 3D representation for stable novel view rendering. We present a feed-forward multi-view reconstruction framework for inverse rendering that directly predicts a structured 3D Gaussian representation with intrinsic material attributes. Each Gaussian primitive is parameterized by mean, normal, opacity, rotation, scale, albedo, metallic, and roughness, enabling a disentangled and physically grounded scene representation. Our model integrates priors from a material estimation network with a multi-view 3D reconstruction backbone, allowing joint prediction of geometry and reflectance parameters in a single forward pass. Experiments on synthetic and real-world datasets demonstrate improved multi-view consistency compared to 2D baselines, accurate material recovery, and stable novel view rendering. Our representation further supports physically-based relighting and more faithful modeling of view-dependent effects compared to existing RGB-based feed-forward reconstruction methods. Our project webpage is: $\href{https://poliik.github.io/invsplat/}{\text{https://poliik.github.io/invsplat/}}$.

Problem

• inverse rendering
• 3D reconstruction
• material estimation
• novel view synthesis
• multi-view consistency

Innovation

• inverse rendering
• 3D Gaussian splatting
• feed-forward reconstruction
• physically-based materials
• multi-view consistency