GlobalSplat: Efficient Feed-Forward 3D Gaussian Splatting via Global Scene Tokens

2026-04-16

AI Summary

Existing 3D Gaussian splatting methods rely on local heuristic strategies, struggling to balance representation compactness, reconstruction speed, and rendering fidelity, while multi-view inputs often introduce redundancy and global inconsistency. This work proposes a “align-then-decode” framework that introduces, for the first time, a global scene token mechanism. By learning a compact implicit scene representation, the method fuses multi-view information and establishes cross-view correspondences prior to explicit decoding. Coupled with a coarse-to-fine training strategy, it effectively suppresses representation inflation without requiring pretrained backbones or dense feature reuse. On RealEstate10K and ACID benchmarks, the approach achieves high-quality novel view synthesis using only 16,000 Gaussians (approximately 4 MB), with single-pass inference as fast as 78 milliseconds.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

The efficient spatial allocation of primitives serves as the foundation of 3D Gaussian Splatting, as it directly dictates the synergy between representation compactness, reconstruction speed, and rendering fidelity. Previous solutions, whether based on iterative optimization or feed-forward inference, suffer from significant trade-offs between these goals, mainly due to the reliance on local, heuristic-driven allocation strategies that lack global scene awareness. Specifically, current feed-forward methods are largely pixel-aligned or voxel-aligned. By unprojecting pixels into dense, view-aligned primitives, they bake redundancy into the 3D asset. As more input views are added, the representation size increases and global consistency becomes fragile. To this end, we introduce GlobalSplat, a framework built on the principle of align first, decode later. Our approach learns a compact, global, latent scene representation that encodes multi-view input and resolves cross-view correspondences before decoding any explicit 3D geometry. Crucially, this formulation enables compact, globally consistent reconstructions without relying on pretrained pixel-prediction backbones or reusing latent features from dense baselines. Utilizing a coarse-to-fine training curriculum that gradually increases decoded capacity, GlobalSplat natively prevents representation bloat. On RealEstate10K and ACID, our model achieves competitive novel-view synthesis performance while utilizing as few as 16K Gaussians, significantly less than required by dense pipelines, obtaining a light 4MB footprint. Further, GlobalSplat enables significantly faster inference than the baselines, operating under 78 milliseconds in a single forward pass. Project page is available at https://r-itk.github.io/globalsplat/

Problem

• 3D Gaussian Splatting
• global scene representation
• spatial allocation
• multi-view consistency
• representation compactness

Innovation

• Global Scene Representation
• Feed-Forward 3D Reconstruction
• Gaussian Splatting
• Cross-View Correspondence
• Compact Latent Encoding