GaussianUpdate: Continual 3D Gaussian Splatting Update for Changing Environments

2025-08-12

AI Summary

To address the challenge of neural rendering models failing to continuously adapt to dynamic environments while suffering from catastrophic forgetting of historical knowledge, this paper proposes a continual learning framework based on 3D Gaussian splatting. Methodologically: (1) a multi-stage incremental update strategy explicitly disentangles and models heterogeneous scene changes—including geometry, appearance, and motion; (2) a vision-perception-driven generative replay mechanism enables self-supervised knowledge consolidation without storing raw images; (3) visibility-aware optimization ensures consistency across both novel and previously observed viewpoints. Evaluated on multiple dynamic NeRF benchmarks, our approach significantly outperforms existing continual learning and neural rendering methods. It supports real-time rendering and fine-grained temporal visualization of scene evolution, achieving— for the first time—high-fidelity, low-forgetting, and lightweight online modeling of dynamic scenes.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Computer Vision: 3D Computer Vision
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Novel view synthesis with neural models has advanced rapidly in recent years, yet adapting these models to scene changes remains an open problem. Existing methods are either labor-intensive, requiring extensive model retraining, or fail to capture detailed types of changes over time. In this paper, we present GaussianUpdate, a novel approach that combines 3D Gaussian representation with continual learning to address these challenges. Our method effectively updates the Gaussian radiance fields with current data while preserving information from past scenes. Unlike existing methods, GaussianUpdate explicitly models different types of changes through a novel multi-stage update strategy. Additionally, we introduce a visibility-aware continual learning approach with generative replay, enabling self-aware updating without the need to store images. The experiments on the benchmark dataset demonstrate our method achieves superior and real-time rendering with the capability of visualizing changes over different times

Problem

• Adapting neural models to scene changes efficiently
• Capturing detailed types of changes over time
• Updating 3D Gaussian representation without extensive retraining

Innovation

• Combines 3D Gaussian with continual learning
• Multi-stage update strategy for scene changes
• Visibility-aware learning with generative replay