LoD-Structured 3D Gaussian Splatting for Streaming Video Reconstruction

2026-01-26

AI Summary

This work addresses the challenges of real-time reconstruction in streaming free-viewpoint video (SFVV) under sparse viewpoints, high training costs, and bandwidth constraints by proposing the StreamLoD-GS framework. The method integrates a hierarchical 3D Gaussian Splatting representation based on anchor points and octrees, a Gaussian mixture model–driven mechanism for separating dynamic and static content, and a quantized residual refinement strategy. Together, these components enable high-quality real-time rendering while substantially reducing storage and computational overhead. Experimental results demonstrate that the proposed approach achieves or surpasses state-of-the-art performance in terms of reconstruction fidelity, optimization efficiency, and memory footprint, making it well-suited for practical streaming SFVV applications.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Time-Series/Data Streams
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Free-Viewpoint Video (FVV) reconstruction enables photorealistic and interactive 3D scene visualization; however, real-time streaming is often bottlenecked by sparse-view inputs, prohibitive training costs, and bandwidth constraints. While recent 3D Gaussian Splatting (3DGS) has advanced FVV due to its superior rendering speed, Streaming Free-Viewpoint Video (SFVV) introduces additional demands for rapid optimization, high-fidelity reconstruction under sparse constraints, and minimal storage footprints. To bridge this gap, we propose StreamLoD-GS, an LoD-based Gaussian Splatting framework designed specifically for SFVV. Our approach integrates three core innovations: 1) an Anchor- and Octree-based LoD-structured 3DGS with a hierarchical Gaussian dropout technique to ensure efficient and stable optimization while maintaining high-quality rendering; 2) a GMM-based motion partitioning mechanism that separates dynamic and static content, refining dynamic regions while preserving background stability; and 3) a quantized residual refinement framework that significantly reduces storage requirements without compromising visual fidelity. Extensive experiments demonstrate that StreamLoD-GS achieves competitive or state-of-the-art performance in terms of quality, efficiency, and storage.

Problem

• Streaming Free-Viewpoint Video
• 3D Gaussian Splatting
• sparse-view reconstruction
• real-time streaming
• storage efficiency

Innovation

• Level-of-Detail (LoD)
• 3D Gaussian Splatting
• Streaming Free-Viewpoint Video
• Motion Partitioning
• Quantized Residual Refinement