Virtualized 3D Gaussians: Flexible Cluster-based Level-of-Detail System for Real-Time Rendering of Composed Scenes

2025-05-10

AI Summary

To address the low real-time rendering efficiency of large-scale 3D Gaussian Splatting (3DGS) scenes—e.g., crowd-level scenes with over 100 million Gaussians—this paper proposes a clustering-based dynamic Level-of-Detail (LOD) system. Methodologically, it constructs a hierarchical Gaussian cluster structure offline; online, it selects critical clusters per-pixel based on visual footprint estimation, incorporates virtualized 3D Gaussian representations, and applies localized splatting to preserve cross-granularity fidelity. A configurable tolerance-aware cluster selection mechanism ensures adaptive detail control. Contributions: This is the first work to achieve stable ≥60 FPS GPU-accelerated rasterization on billion-Gaussian scenes. In both synthetic and real-world scenes containing humans, trees, and buildings, visual reconstruction error remains strictly below the user-specified tolerance. The method significantly enhances real-time performance and visual consistency for large-scale, composable, interactive 3D applications.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Clustering
• Humans and AI: Crowd Sourcing and Human Computation

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

3D Gaussian Splatting (3DGS) enables the reconstruction of intricate digital 3D assets from multi-view images by leveraging a set of 3D Gaussian primitives for rendering. Its explicit and discrete representation facilitates the seamless composition of complex digital worlds, offering significant advantages over previous neural implicit methods. However, when applied to large-scale compositions, such as crowd-level scenes, it can encompass numerous 3D Gaussians, posing substantial challenges for real-time rendering. To address this, inspired by Unreal Engine 5's Nanite system, we propose Virtualized 3D Gaussians (V3DG), a cluster-based LOD solution that constructs hierarchical 3D Gaussian clusters and dynamically selects only the necessary ones to accelerate rendering speed. Our approach consists of two stages: (1) Offline Build, where hierarchical clusters are generated using a local splatting method to minimize visual differences across granularities, and (2) Online Selection, where footprint evaluation determines perceptible clusters for efficient rasterization during rendering. We curate a dataset of synthetic and real-world scenes, including objects, trees, people, and buildings, each requiring 0.1 billion 3D Gaussians to capture fine details. Experiments show that our solution balances rendering efficiency and visual quality across user-defined tolerances, facilitating downstream interactive applications that compose extensive 3DGS assets for consistent rendering performance.

Problem

• Addresses real-time rendering challenges in large-scale 3D Gaussian scenes
• Proposes cluster-based LOD system to optimize rendering speed and quality
• Enables efficient composition of complex 3D assets with minimal visual loss

Innovation

• Virtualized 3D Gaussians for scalable real-time rendering
• Hierarchical cluster-based LOD system for dynamic selection
• Offline build and online selection for efficient rasterization