MTGS: Multi-Traversal Gaussian Splatting

2025-03-16

AI Summary

To address degraded driving scene reconstruction quality caused by dynamic object interference and cross-pass appearance inconsistency in multi-pass vehicle-mounted data collection, this paper proposes a multi-pass dynamic scene reconstruction method tailored for novel-view synthesis. Our approach explicitly decouples static geometry (shared across passes), dynamic objects (independently modeled per pass), and appearance discrepancies (corrected via learnable spherical harmonic residual functions) within a multi-pass dynamic scene graph structure. We integrate Gaussian splatting rendering to achieve efficient, high-fidelity novel-view synthesis. Evaluated on the nuPlan dataset, our method reduces LPIPS by 23.5% and improves geometric accuracy by 46.3% over single-pass baselines. It enables free-viewpoint navigation and significantly enhances fidelity and generalization for downstream applications such as autonomous driving simulators.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Computer Vision: Scene Analysis & Understanding
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Multi-traversal data, commonly collected through daily commutes or by self-driving fleets, provides multiple viewpoints for scene reconstruction within a road block. This data offers significant potential for high-quality novel view synthesis, which is crucial for applications such as autonomous vehicle simulators. However, inherent challenges in multi-traversal data often result in suboptimal reconstruction quality, including variations in appearance and the presence of dynamic objects. To address these issues, we propose Multi-Traversal Gaussian Splatting (MTGS), a novel approach that reconstructs high-quality driving scenes from arbitrarily collected multi-traversal data by modeling a shared static geometry while separately handling dynamic elements and appearance variations. Our method employs a multi-traversal dynamic scene graph with a shared static node and traversal-specific dynamic nodes, complemented by color correction nodes with learnable spherical harmonics coefficient residuals. This approach enables high-fidelity novel view synthesis and provides flexibility to navigate any viewpoint. We conduct extensive experiments on a large-scale driving dataset, nuPlan, with multi-traversal data. Our results demonstrate that MTGS improves LPIPS by 23.5% and geometry accuracy by 46.3% compared to single-traversal baselines. The code and data would be available to the public.

Problem

• Reconstructs high-quality driving scenes from multi-traversal data.
• Handles dynamic objects and appearance variations separately.
• Improves novel view synthesis and geometry accuracy significantly.

Innovation

• MTGS models shared static geometry separately
• Uses dynamic scene graph with color correction
• Improves LPIPS and geometry accuracy significantly