MapGS: Generalizable Pretraining and Data Augmentation for Online Mapping via Novel View Synthesis

2025-01-11

AI Summary

To address the weak cross-sensor generalization (due to intrinsic/extrinsic parameter variations) and poor model universality in online autonomous driving mapping, this paper introduces differentiable 3D Gaussian splatting into an online mapping pretraining framework—the first such application. It synthesizes target-sensor-view images and semantic labels via neural rendering to learn sensor-agnostic universal representations. We further propose a closed-loop annotation mapping mechanism grounded in rendered data, jointly enforced by cross-configuration geometric consistency constraints to enable zero-shot transfer. Evaluated on nuScenes and Argoverse 2, our method improves mapping accuracy by 18% over prior work; remarkably, it surpasses state-of-the-art methods using only 25% of the original annotations. Moreover, training convergence is accelerated, and reliance on manual annotation is significantly reduced.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Intelligent Robots: Localization, Mapping, and Navigation
• Machine Learning: Neuro-Symbolic Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Online mapping reduces the reliance of autonomous vehicles on high-definition (HD) maps, significantly enhancing scalability. However, recent advancements often overlook cross-sensor configuration generalization, leading to performance degradation when models are deployed on vehicles with different camera intrinsics and extrinsics. With the rapid evolution of novel view synthesis methods, we investigate the extent to which these techniques can be leveraged to address the sensor configuration generalization challenge. We propose a novel framework leveraging Gaussian splatting to reconstruct scenes and render camera images in target sensor configurations. The target config sensor data, along with labels mapped to the target config, are used to train online mapping models. Our proposed framework on the nuScenes and Argoverse 2 datasets demonstrates a performance improvement of 18% through effective dataset augmentation, achieves faster convergence and efficient training, and exceeds state-of-the-art performance when using only 25% of the original training data. This enables data reuse and reduces the need for laborious data labeling. Project page at https://henryzhangzhy.github.io/mapgs.

Problem

• Sensor Compatibility
• Mapping Precision
• Model Generalization

Innovation

• MapGS Framework
• Gaussian Splattering
• Cross-Sensor Generalization