Compact Feed-Forward 3D Gaussians via Saliency-Guided Primitive Merging

2026-08-11

AI Summary

This work addresses the redundancy and inefficiency inherent in feed-forward 3D Gaussian splatting methods, which generate primitives on a per-pixel basis. To overcome this limitation, the authors propose a structure-aware primitive merging approach that leverages saliency-guided adaptive superpixel segmentation to achieve spatially and appearance-consistent clustering. The method is integrated into a learnable encoder-merger-multi-resolution decoder architecture, enabling flexible trade-offs between rendering quality and computational efficiency. The proposed framework can compress Gaussian primitives produced by any feed-forward method to as few as 1/20 of their original count while preserving high-fidelity rendering. Extensive evaluations demonstrate that this approach significantly outperforms existing compression techniques in terms of reconstruction accuracy and robustness, while also enabling highly efficient rendering.

Abstract

3D scene reconstruction, modeling, and rendering are highly relevant for numerous tasks, and 3D Gaussian splatting has become a standard choice in this context. Its feed-forward variants provide fast reconstruction from sparse input views but often produce per-pixel primitives, leading to highly redundant and thus inefficient representations. We present a structure-aware merging pipeline that takes per-pixel primitives from any feed-forward method and consolidates them into a compact, content-adaptive Gaussian set while largely retaining visual quality at just $\frac{1}{20}^\text{th}$ of the Gaussians of a per-pixel method. We group spatially coherent Gaussians of similar appearance into variable-size clusters via adaptive superpixel segmentation guided by a saliency map, which allocates fine segments to textured regions and coarse segments to homogeneous areas. We compress each cluster into a compact latent representation through a learned encoder, then match and consolidate representations across views based on geometric overlap and feature similarity via a learned merger. A level-of-detail decoder then produces the final Gaussians at a controllable resolution, enabling a flexible quality-efficiency trade-off at inference. As a post-processing module, the pipeline is backbone-agnostic, leveraging the strengths of existing feed-forward methods. This leads to better and more robust quality than achieved by previous approaches that target a reduction in primitive count, while providing a highly compact representation, that can be rendered efficiently.

Problem

• 3D Gaussian splatting
• feed-forward reconstruction
• primitive redundancy
• compact representation
• scene reconstruction

Innovation

• saliency-guided merging
• compact 3D Gaussians
• adaptive superpixel segmentation
• learned primitive consolidation
• level-of-detail decoding