Learning Stable Canonical Worlds for Novel View Synthesis and Beyond

2026-06-22

AI Summary

Existing feedforward Gaussian splatting methods struggle to construct stable scene representations as the number of multi-view inputs increases, often suffering from noise and redundant information. To address this limitation, this work proposes CanonicalGS, a framework that maps multi-view observations into a scene-centric canonical implicit space. It introduces an uncertainty-guided evidential fusion mechanism that jointly aggregates depth, semantic features, and uncertainty evidence to effectively suppress unreliable observations. The resulting representation is both scalable and transferable across scenes. Experimental results demonstrate significant improvements in novel view synthesis, with PSNR gains of up to 2.5 dB, and an 11% increase in semantic segmentation accuracy.

Abstract

Feed-forward Gaussian splatting (FFGS) facilitates real-time novel view synthesis, yet current methods often remain tied to view-dependent predictions. As more input views are added, they may accumulate noisy or redundant evidence instead of converging to a stable scene representation. In this paper, we introduce CanonicalGS, a feed-forward pipeline that maps cluttered multi-view observations into a stable, scene-centric representation. CanonicalGS first extracts view-centric evidence from depth, semantic features, and uncertainty estimates, and then aggregates this evidence in a canonical latent world using uncertainty-aware fusion. By emphasizing reliable observations while suppressing uncertain or redundant ones, CanonicalGS produces representations that scale more effectively for novel view synthesis and transfer to downstream visual perception tasks. Experiments show up to a $2.5$ dB improvement in peak signal-to-noise ratio for synthesizing novel views and an $11\%$ gain in semantic segmentation accuracy.

Problem

• novel view synthesis
• stable scene representation
• view-dependent prediction
• multi-view observations
• canonical world

Innovation

• CanonicalGS
• feed-forward Gaussian splatting
• uncertainty-aware fusion
• novel view synthesis
• canonical scene representation