CoSAG: Compact Semantic Anchor Gaussians via Training-Free Rate-Distortion Coding

2026-07-11

AI Summary

This work addresses the challenge of gigabyte-scale scene bloat in open-vocabulary 3D scene understanding caused by attaching high-dimensional semantic features to 3D Gaussians. To enable efficient deployment, we propose decoupling semantic field construction from storage: leveraging training-free transmittance-weighted feature enhancement, spatially anchored semantic anchors, and multi-view consistency-based denoising to build the semantic field, alongside a novel decoder-free spatial prediction entropy encoder that efficiently compresses Gaussian–anchor binding relationships. For the first time, we ground semantic anchors in 3D space to enhance predictability, achieving sub-megabyte scene sizes without per-scene training—37–76× smaller than LangSplatV2 while yielding superior accuracy. Our method matches or surpasses state-of-the-art performance across 2D rendering, 3D selection, and dense LSeg tasks.

Abstract

Open-vocabulary 3D scene understanding is commonly achieved by embedding 2D vision-language features such as CLIP into a 3D Gaussian Splatting scene, turning it into a text-queryable semantic field. However, attaching a high-dimensional feature to each of millions of Gaussians inflates a single scene to gigabytes, which makes storage and deployment the real bottleneck of these fields. Existing compact methods each learn and ship a per-scene codec, an autoencoder, a quantized codebook, or a distilled feature field, entangling field construction with field storage and never compressing the per-Gaussian assignment that holds the bulk of the cost. We argue that construction and storage should be decoupled, and that storage is a rate-distortion problem over the per-Gaussian binding to a small anchor table, a structure no prior open-vocabulary method compresses. We present CoSAG, which constructs the field without any per-scene training through a closed-form transmittance-weighted lift, spatially grounded semantic anchors, and multi-view denoising, and stores it with a spatially predictive entropy coder that ships no decoder. Because the anchors are spatially grounded, the binding is predictable and therefore highly compressible. The transmittance-weighted lift and multi-view denoising yield a clean, view-consistent assignment, so the entropy coder spends almost no rate on correcting noise and instead codes only the residual against its spatial prediction. CoSAG reaches sub-megabyte storage while matching or exceeding the state of the art across the 2D-rendered, 3D-selection, and dense-LSeg protocols, reducing field size by 37 to 76x relative to LangSplatV2 at higher accuracy.

Problem

• open-vocabulary 3D scene understanding
• 3D Gaussian Splatting
• semantic feature compression
• storage bottleneck
• rate-distortion coding

Innovation

• Gaussian Splatting
• Rate-Distortion Coding
• Semantic Anchors
• Training-Free Compression
• Open-Vocabulary 3D