CAGS: Color-Adaptive Volumetric Video Streaming with Dynamic 3D Gaussian Splatting

2026-05-09

AI Summary

This work addresses the challenge of achieving real-time performance, visual fidelity, and level-of-detail (LoD) adaptability simultaneously in 3D Gaussian splatting video streaming under bandwidth constraints, where conventional LoD strategies often introduce color distortion. To overcome these limitations, we propose a novel multi-level LoD framework based on vector quantization (VQ), incorporating a color-adaptive mechanism into Gaussian splatting streaming for the first time. Our approach leverages low-resolution reference images generated on the server side to correct color deviations caused by compression at the client. By moving beyond traditional density-driven LoD paradigms and supporting diverse Gaussian representations, the method achieves a 5–20 dB PSNR improvement over existing systems under varying bandwidth conditions, while maintaining low latency, high visual fidelity, and strong generalization capability.

Abstract

Volumetric video (VV) streaming enables real-time, immersive access to remote 3D environments, powering telepresence, ecological monitoring, and robotic teleoperation. These applications turn VV streaming into a real-time interface to remote physical environments, imposing new system-level demands for photorealistic scene representation, low-latency interaction, and robust performance under heterogeneous networks. 3D Gaussian Splatting (3DGS) has been widely used for real-time photorealistic rendering, offering superior visual quality and rendering performance, but it faces challenges due to bandwidth consumption. Furthermore, as the foundation of adaptive VV streaming, existing Levels of Detail (LoD) methods based on density are not well-suited to Gaussian representations, leading to visible gaps and severe quality degradation. Recent studies have also explored attribute compression techniques to reduce bandwidth consumption. Our preliminary studies reveal that aggressive attribute compression primarily causes color distortion, which can be effectively corrected in the rendered image using a reference image. Motivated by these findings, we propose a novel Color-Adaptive scheme for adaptive VV streaming that uses vector quantization (VQ) to establish LoDs and correct color distortions with low-resolution reference images. We further present CAGS, an adaptive VV streaming system compatible with diverse Gaussian representations, which integrates the Color-Adaptive scheme by rendering reference images on the streaming server and performing color restoration on the client. Extensive experiments on our prototype system demonstrate that CAGS outperforms the existing adaptive streaming systems in PSNR by 5$\sim$20 dB under fluctuating bandwidth, operates significantly faster than existing scalable Gaussian compression methods, and generalizes across different Gaussian representations.

Problem

• Volumetric Video Streaming
• 3D Gaussian Splatting
• Color Distortion
• Levels of Detail
• Bandwidth Consumption

Innovation

• Color-Adaptive Streaming
• 3D Gaussian Splatting
• Vector Quantization
• Volumetric Video
• Attribute Compression