GStex: Per-Primitive Texturing of 2D Gaussian Splatting for Decoupled Appearance and Geometry Modeling

2024-09-19

arXiv.org

AI Summary

Existing 2D Gaussian splatting tightly couples appearance and geometry within each Gaussian primitive, leading to inefficient texture modeling—even planar scenes require excessive primitives. This work proposes a decoupled modeling framework: each 2D Gaussian is assigned an independent, differentiable texture map, enabling the first primitive-level texture parameterization. By introducing UV coordinate mapping and jointly optimizing geometric and texture parameters, appearance representation is fully disentangled from geometric topology constraints. The method supports efficient retexturing, real-time editing, and cross-scene texture transfer. Experiments demonstrate that our approach achieves superior view synthesis quality and significantly improved planar texture fidelity—particularly in regions with complex textures—while using substantially fewer Gaussians than baseline methods.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Mixed Discrete/Continuous Search
• Planning, Routing, and Scheduling: Mixed Discrete/Continuous Planning

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Gaussian splatting has demonstrated excellent performance for view synthesis and scene reconstruction. The representation achieves photorealistic quality by optimizing the position, scale, color, and opacity of thousands to millions of 2D or 3D Gaussian primitives within a scene. However, since each Gaussian primitive encodes both appearance and geometry, these attributes are strongly coupled--thus, high-fidelity appearance modeling requires a large number of Gaussian primitives, even when the scene geometry is simple (e.g., for a textured planar surface). We propose to texture each 2D Gaussian primitive so that even a single Gaussian can be used to capture appearance details. By employing per-primitive texturing, our appearance representation is agnostic to the topology and complexity of the scene's geometry. We show that our approach, GStex, yields improved visual quality over prior work in texturing Gaussian splats. Furthermore, we demonstrate that our decoupling enables improved novel view synthesis performance compared to 2D Gaussian splatting when reducing the number of Gaussian primitives, and that GStex can be used for scene appearance editing and re-texturing.

Problem

• Decouples appearance and geometry modeling
• Enhances visual quality with per-primitive texturing
• Improves novel view synthesis with fewer primitives

Innovation

• Per-primitive texturing for Gaussians
• Decouples appearance and geometry modeling
• Enhances novel view synthesis performance