GSStream: 3D Gaussian Splatting based Volumetric Scene Streaming System

2026-03-10

AI Summary

This work addresses the challenge of real-time streaming for 3D Gaussian Splatting (3DGS), a high-quality volumetric rendering technique hindered by its substantial data volume and bandwidth demands. To this end, the authors propose GSStream, a novel streaming system that first constructs the first viewport trajectory dataset tailored for volumetric rendering and leverages collaborative priors from multiple users along with historical behavior to enable accurate viewport prediction. Building upon this, GSStream introduces a deep reinforcement learning–driven bitrate adaptation mechanism that dynamically adjusts to evolving state and action spaces. Experimental results demonstrate that GSStream significantly outperforms existing volumetric rendering streaming approaches in both visual quality and network resource efficiency.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Time-Series/Data Streams
• Data Mining & Knowledge Management: Data Stream Mining

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Recently, the 3D Gaussian splatting (3DGS) technique for real-time radiance field rendering has revolutionized the field of volumetric scene representation, providing users with an immersive experience. But in return, it also poses a large amount of data volume, which is extremely bandwidth-intensive. Cutting-edge researchers have tried to introduce different approaches and construct multiple variants for 3DGS to obtain a more compact scene representation, but it is still challenging for real-time distribution. In this paper, we propose GSStream, a novel volumetric scene streaming system to support 3DGS data format. Specifically, GSStream integrates a collaborative viewport prediction module to better predict users' future behaviors by learning collaborative priors and historical priors from multiple users and users' viewport sequences and a deep reinforcement learning (DRL)-based bitrate adaptation module to tackle the state and action space variability challenge of the bitrate adaptation problem, achieving efficient volumetric scene delivery. Besides, we first build a user viewport trajectory dataset for volumetric scenes to support the training and streaming simulation. Extensive experiments prove that our proposed GSStream system outperforms existing representative volumetric scene streaming systems in visual quality and network usage. Demo video: https://youtu.be/3WEe8PN8yvA.

Problem

• 3D Gaussian Splatting
• volumetric scene streaming
• bandwidth-intensive
• real-time distribution
• data volume

Innovation

• 3D Gaussian Splatting
• volumetric scene streaming
• viewport prediction
• deep reinforcement learning
• bitrate adaptation