L3GS: Layered 3D Gaussian Splats for Efficient 3D Scene Delivery

2025-04-07

AI Summary

To address the challenges of high bandwidth consumption, large first-frame latency, and compromised rendering quality in client-side real-time rendering of 3D Gaussian Splatting (3DGS) scenes, this paper proposes Layered 3D Gaussian Splatting (L3GS). Our method introduces: (1) a hierarchical 3DGS representation amenable to layered encoding; (2) an adaptive, progressive download scheduling algorithm guided by user gaze trajectories and SSIM-based perceptual quality metrics; and (3) an end-to-end optimized transmission pipeline compatible with mainstream 3DGS compression formats. Evaluated on VR traces, L3GS achieves an average SSIM improvement of 16.9%, significantly reduces per-scene transmission data volume, cuts first-frame latency by 42%, and maintains high-fidelity rendering quality—effectively balancing bandwidth efficiency, responsiveness, and visual fidelity.

Technology Category

• Computer Vision: 3D Computer Vision
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Traditional 3D content representations include dense point clouds that consume large amounts of data and hence network bandwidth, while newer representations such as neural radiance fields suffer from poor frame rates due to their non-standard volumetric rendering pipeline. 3D Gaussian splats (3DGS) can be seen as a generalization of point clouds that meet the best of both worlds, with high visual quality and efficient rendering for real-time frame rates. However, delivering 3DGS scenes from a hosting server to client devices is still challenging due to high network data consumption (e.g., 1.5 GB for a single scene). The goal of this work is to create an efficient 3D content delivery framework that allows users to view high quality 3D scenes with 3DGS as the underlying data representation. The main contributions of the paper are: (1) Creating new layered 3DGS scenes for efficient delivery, (2) Scheduling algorithms to choose what splats to download at what time, and (3) Trace-driven experiments from users wearing virtual reality headsets to evaluate the visual quality and latency. Our system for Layered 3D Gaussian Splats delivery L3GS demonstrates high visual quality, achieving 16.9% higher average SSIM compared to baselines, and also works with other compressed 3DGS representations.

Problem

• Reducing high network data consumption in 3DGS scene delivery
• Improving visual quality and latency for 3D content streaming
• Optimizing splat download scheduling for efficient 3D rendering

Innovation

• Layered 3DGS scenes for efficient delivery
• Scheduling algorithms for splat downloads
• Trace-driven VR experiments for evaluation