ReconSplat: Generalizable 3D Scene Reconstruction Beyond Observed Views

2026-08-28

AI Summary

本文提出ReconSplat模型，通过结合3D高斯点云和多视图潜在扩散模型解决未观测区域的合理视角生成与几何一致性问题。

Abstract

We introduce ReconSplat, a feed-forward model for 3D scene reconstruction that aims to address the longstanding trade-off between plausible view generation for unobserved regions and geometric consistency, providing both geometrically aligned novel views and sharp depth estimates. Our approach builds on 3D Gaussian splatting (3DGS) as an intermediate differentiable scene representation and integrates it with a multi-view latent diffusion model (MV-LDM) trained to act simultaneously as a refiner and an inpainter for appearance and scene geometry. We enforce geometric consistency by guiding the diffusion process with variational 3D latent features for appearance and geometry, encoded by the feed-forward 3DGS representation and rasterized to 2D latent space. ReconSplat produces both photorealistic novel views and accurate depth maps on real-world benchmarks, RealEstate10K and DL3DV-10K, outperforming existing methods in challenging extrapolation setups. Notably, ReconSplat allows the extrapolation of unseen and challenging viewpoints jointly with coherent and precise scene geometry.

Problem

• 3D Scene Reconstruction
• Geometric Consistency
• Novel View Generation

Innovation

• ReconSplat
• 3D Gaussian Splatting
• Multi-view Latent Diffusion Model
• Geometric Consistency
• Novel View Synthesis