4DGCPro: Efficient Hierarchical 4D Gaussian Compression for Progressive Volumetric Video Streaming

2025-09-22

AI Summary

To address the conflict between network/device heterogeneity and real-time decoding on mobile devices in 4D volumetric video streaming, this paper proposes a hierarchical 4D Gaussian compression framework. Our method introduces: (1) a perception-weighted and motion-aware adaptive grouping scheme for hierarchical 4D Gaussian representation, enabling progressive quality- and bitrate-scalability from a single bitstream; and (2) end-to-end entropy-constrained optimization, jointly incorporating per-layer rate-distortion supervision and attribute-specific entropy modeling to achieve hierarchical rate-distortion optimization. Extensive experiments on multiple benchmark datasets demonstrate significant improvements over state-of-the-art methods. A single trained model supports flexible, runtime adjustment of reconstruction quality and bitrate. Moreover, the framework enables high-fidelity real-time decoding and rendering on resource-constrained mobile platforms.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Learning on the Edge & Model Compression
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

Achieving seamless viewing of high-fidelity volumetric video, comparable to 2D video experiences, remains an open challenge. Existing volumetric video compression methods either lack the flexibility to adjust quality and bitrate within a single model for efficient streaming across diverse networks and devices, or struggle with real-time decoding and rendering on lightweight mobile platforms. To address these challenges, we introduce 4DGCPro, a novel hierarchical 4D Gaussian compression framework that facilitates real-time mobile decoding and high-quality rendering via progressive volumetric video streaming in a single bitstream. Specifically, we propose a perceptually-weighted and compression-friendly hierarchical 4D Gaussian representation with motion-aware adaptive grouping to reduce temporal redundancy, preserve coherence, and enable scalable multi-level detail streaming. Furthermore, we present an end-to-end entropy-optimized training scheme, which incorporates layer-wise rate-distortion (RD) supervision and attribute-specific entropy modeling for efficient bitstream generation. Extensive experiments show that 4DGCPro enables flexible quality and multiple bitrate within a single model, achieving real-time decoding and rendering on mobile devices while outperforming existing methods in RD performance across multiple datasets. Project Page: https://mediax-sjtu.github.io/4DGCPro

Problem

• Achieving seamless high-fidelity volumetric video streaming comparable to 2D video
• Enabling flexible quality adjustment and multiple bitrates within single model
• Supporting real-time decoding and rendering on lightweight mobile platforms

Innovation

• Hierarchical 4D Gaussian representation with adaptive grouping
• End-to-end entropy-optimized training with RD supervision
• Single model enabling progressive streaming and real-time decoding