LongSplat: Online Generalizable 3D Gaussian Splatting from Long Sequence Images

2025-07-21

AI Summary

Existing 3D Gaussian Splatting methods struggle to simultaneously achieve real-time performance and temporal stability under long-sequence, streaming image inputs. To address this, we propose the first online adaptive 3D Gaussian reconstruction framework. Our approach introduces two key innovations: (1) Gaussian Image Representation (GIR), a structured 2D feature map encoding of 3D Gaussian parameters, enabling efficient spatiotemporal fusion and identity-aware redundancy compression; and (2) a lightweight streaming update mechanism that supports incremental integration of novel views while dynamically pruning outdated information. Experiments demonstrate that our method achieves real-time reconstruction (>30 FPS) without compromising high-fidelity novel-view synthesis quality. Moreover, it reduces the total number of Gaussians by 44%, significantly improving the trade-off between computational efficiency and storage compactness.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Time-Series/Data Streams

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

3D Gaussian Splatting achieves high-fidelity novel view synthesis, but its application to online long-sequence scenarios is still limited. Existing methods either rely on slow per-scene optimization or fail to provide efficient incremental updates, hindering continuous performance. In this paper, we propose LongSplat, an online real-time 3D Gaussian reconstruction framework designed for long-sequence image input. The core idea is a streaming update mechanism that incrementally integrates current-view observations while selectively compressing redundant historical Gaussians. Crucial to this mechanism is our Gaussian-Image Representation (GIR), a representation that encodes 3D Gaussian parameters into a structured, image-like 2D format. GIR simultaneously enables efficient fusion of current-view and historical Gaussians and identity-aware redundancy compression. These functions enable online reconstruction and adapt the model to long sequences without overwhelming memory or computational costs. Furthermore, we leverage an existing image compression method to guide the generation of more compact and higher-quality 3D Gaussians. Extensive evaluations demonstrate that LongSplat achieves state-of-the-art efficiency-quality trade-offs in real-time novel view synthesis, delivering real-time reconstruction while reducing Gaussian counts by 44% compared to existing per-pixel Gaussian prediction methods.

Problem

• Enables online 3D Gaussian reconstruction for long sequences
• Reduces memory and computational costs via redundancy compression
• Improves efficiency-quality trade-off in real-time view synthesis

Innovation

• Streaming update mechanism for incremental integration
• Gaussian-Image Representation for efficient fusion
• Image compression for compact high-quality Gaussians