GS-SDF: LiDAR-Augmented Gaussian Splatting and Neural SDF for Geometrically Consistent Rendering and Reconstruction

2025-03-13

AI Summary

Addressing the challenge of high-fidelity rendering and geometrically consistent reconstruction from sparse LiDAR data in digital twins, this paper proposes an SDF-guided Gaussian splatting framework. It models manifold geometry priors via a neural signed distance field (SDF), enabling SDF-constrained Gaussian initialization and geometric consistency regularization. By integrating LiDAR-visual joint optimization with multi-view photometric consistency loss, the method overcomes limitations of purely rendering-driven approaches in complex scenes. Notably, it is the first to combine physically interpretable voxel placement with Gaussian rasterization. Evaluated across diverse trajectories, the framework achieves significant improvements: a 12.6% increase in surface reconstruction F-Score and a 2.1 dB gain in rendering PSNR, while enabling real-time, high-geometric-fidelity rendering.

Technology Category

• Computer Vision: 3D Computer Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT

Abstract

Digital twins are fundamental to the development of autonomous driving and embodied artificial intelligence. However, achieving high-granularity surface reconstruction and high-fidelity rendering remains a challenge. Gaussian splatting offers efficient photorealistic rendering but struggles with geometric inconsistencies due to fragmented primitives and sparse observational data in robotics applications. Existing regularization methods, which rely on render-derived constraints, often fail in complex environments. Moreover, effectively integrating sparse LiDAR data with Gaussian splatting remains challenging. We propose a unified LiDAR-visual system that synergizes Gaussian splatting with a neural signed distance field. The accurate LiDAR point clouds enable a trained neural signed distance field to offer a manifold geometry field, This motivates us to offer an SDF-based Gaussian initialization for physically grounded primitive placement and a comprehensive geometric regularization for geometrically consistent rendering and reconstruction. Experiments demonstrate superior reconstruction accuracy and rendering quality across diverse trajectories. To benefit the community, the codes will be released at https://github.com/hku-mars/GS-SDF.

Problem

• Achieving high-granularity surface reconstruction and rendering.
• Addressing geometric inconsistencies in Gaussian splatting.
• Integrating sparse LiDAR data with Gaussian splatting effectively.

Innovation

• Unified LiDAR-visual system integration
• Neural SDF for manifold geometry field
• SDF-based Gaussian initialization for consistency