Filling the Unseen: Scene Extrapolation via 3D Gaussian Splatting

2026-09-06

AI Summary

该研究针对3D高斯点绘在未观察区域的重建问题，提出了一种新的框架，通过独立视图检测机制和质量感知掩模模块提高场景外推的质量。

Abstract

3D Gaussian Splatting achieves photorealistic reconstruction within training view distribution, yet it degrades on out-of-distribution novel views, exhibiting holes in unobserved regions and artifacts in observable areas. Recent works formulate this task as extrapolation and interpolation and try to address it with generative models, but remain limited in extrapolation scale and quality. They repeat a generate-reconstruct-shift cycle to progressively build a scene, which introduces accumulated errors with every step conditioning on previous outcomes. In this work, we propose a holistic framework for extrapolation and interpolation. We devise an independent camera view detection mechanism to enable parallel conflict-free extrapolation, circumventing the reliance on the aforementioned error-prone cycle. Building upon this, we design a hierarchical pipeline that extrapolates independent and dependent camera views separately. Additionally, previous methods overlook inconsistency between generated and original images, resulting in compromising well-reconstructed areas. We propose a plug-and-play Quality-Aware Mask (QA-Mask) module, enabling selective utilization of generated data. By calibrating learning weights with pixel-wise rendering quality, it prevents generation-induced degradations on well-constructed areas. Extensive experiments demonstrate the superior performance of our framework, with QA-Mask generalizing to multiple generative reconstruction models.

Problem

• 3D Gaussian Splatting
• out-of-distribution novel views
• extrapolation and interpolation
• accumulated errors
• generative models

Innovation

• independent camera view detection
• hierarchical pipeline
• Quality-Aware Mask (QA-Mask)
• parallel conflict-free extrapolation