SplatGuide: Geometric Priors from 3D Gaussians for Pose-Free Novel View Synthesis

2026-08-17

AI Summary

This study addresses the disconnection between geometric understanding and content generation in pose-free novel view synthesis by proposing a unified framework based on single-view 3D Gaussian Splatting. By leveraging three complementary signals—rendered images, visibility voting, and reconstruction tokens—the method provides pixel-level geometric conditioning and occlusion-aware references to effectively bridge information gaps and guide multi-view diffusion models. Extensive experiments demonstrate that this approach achieves state-of-the-art performance across multiple benchmarks. Notably, it surpasses ground-truth pose baselines on RealEstate10K, significantly enhancing both synthesis quality and geometric consistency in pose-free scenarios. These results underscore the efficacy of integrating explicit geometric priors within generative frameworks for robust novel view synthesis without camera pose supervision.

Abstract

Generating photorealistic novel views from unposed images requires both 3D geometric understanding and the ability to synthesize unseen content. A natural strategy combines feed-forward 3DGS reconstruction with multi-view diffusion. Yet prior pipelines extract at most one signal from the reconstruction, either pixel rendering or learned features, while none exploits per-Gaussian visibility for occlusion-aware reference selection. This *information disconnect* leaves renderable geometry, visibility cues, and learned features unused. SplatGuide closes this disconnect by reusing a single 3DGS scene across three complementary roles. Rendered images provide pixel-aligned geometric conditioning. Per-Gaussian source-view indices are rendered into a target-view voting map for occlusion-aware reference selection. Reconstruction tokens supply feature-level guidance via cross-attention. All three signals derive from the same reconstruction forward pass. Across RealEstate10K, DL3DV, Tanks-and-Temples, and Mip-NeRF 360, SplatGuide achieves state-of-the-art pose-free novel view synthesis. On RealEstate10K, with a moderate number of input views, it surpasses the ground-truth-pose baseline.

Problem

• Pose-Free Novel View Synthesis
• 3D Gaussian Splatting
• Information Disconnect
• Occlusion-Aware Reference Selection

Innovation

• Pose-Free Novel View Synthesis
• 3D Gaussian Splatting
• Occlusion-Aware Reference Selection
• Geometric Priors
• Multi-View Diffusion