From Sparse and Imperfect 2D Anchors to Consistent 3D Gaussian Street Scenes: Support-Aware Appearance

2026-06-24

AI Summary

This work addresses the challenges of view inconsistency, noise propagation, and deployment compatibility in generating consistent, high-quality 3D Gaussian street scenes from sparse and imperfect 2D editing anchors. The authors propose a support-aware coarse-to-fine optimization framework that models appearance residuals via teacher–student distillation, jointly performs frequency decomposition and confidence estimation in renderer space, and introduces support-aware Gaussian spatial aggregation along with spherical harmonic coefficient baking. This approach uniquely unifies handling of sparse inputs, noise suppression, and compatibility with standard rasterizers. Evaluated on datasets such as Waymo and Tanks and Temples, the method significantly outperforms existing edit-driven baselines, achieving an excellent balance among cross-view consistency, target alignment, content preservation, and artifact suppression.

Abstract

Image priors can synthesize target conditions for 3D Gaussian street scenes, but independently edited views do not define a coherent 3D target. Direct fitting can propagate view-specific noise, while existing pipelines do not jointly handle imperfect sparse anchors and standard-rasterizer deployment. To address this gap, teacher-relative appearance residual distillation is introduced for appearance baking. A structured space for frequency decomposition, confidence estimation, and primitive-level lifting is formed by residuals between teacher anchors and original renders. The direct optimization signal is supplied by renderer-space matching, while primitive assignment is regularized by support-aware Gaussian-space aggregation. Supported detail is admitted and unsupported noise is suppressed through confidence-gated coarse-to-fine optimization, after which all residuals are baked into fixed-geometry spherical-harmonic coefficients. The teacher and auxiliary training modules are discarded at inference. Evaluation across Waymo street assets, Tanks and Temples scenes, and multiple target conditions shows a favorable overall balance of target alignment, content preservation, artifact suppression, and cross-view consistency over editing-based baselines. Ablations confirm the effectiveness of the main components. Code will be released at https://github.com/Cagares/Baking-for-3D-Gaussian.

Problem

• 3D Gaussian
• sparse anchors
• cross-view consistency
• appearance baking
• street scenes

Innovation

• 3D Gaussian Splatting
• Appearance Baking
• Support-Aware Aggregation
• Residual Distillation
• Cross-View Consistency