EGG-Fusion: Efficient 3D Reconstruction with Geometry-aware Gaussian Surfel on the Fly

2025-12-01

AI Summary

To address the trade-off among geometric accuracy, noise robustness, and computational efficiency in real-time 3D reconstruction, this paper proposes a geometry-aware sparse-to-dense Gaussian surfel reconstruction framework. Methodologically, it incorporates information filtering to explicitly model sensor noise and introduces a geometry-constrained Gaussian surfel mapping module that jointly optimizes multi-view consistency and differentiable rendering atop a 3D Gaussian lattice representation, enabling noise-aware real-time fusion. Evaluated on Replica and ScanNet++, the method achieves a surface reconstruction error of 0.6 cm—over 20% improvement over state-of-the-art Gaussian SLAM approaches—while sustaining real-time performance at 24 FPS. Its core contribution lies in the first integration of information filtering into a differentiable Gaussian SLAM pipeline, thereby simultaneously enhancing geometric fidelity and noise robustness.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Computer Vision: 3D Computer Vision
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Real-time 3D reconstruction is a fundamental task in computer graphics. Recently, differentiable-rendering-based SLAM system has demonstrated significant potential, enabling photorealistic scene rendering through learnable scene representations such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS). Current differentiable rendering methods face dual challenges in real-time computation and sensor noise sensitivity, leading to degraded geometric fidelity in scene reconstruction and limited practicality. To address these challenges, we propose a novel real-time system EGG-Fusion, featuring robust sparse-to-dense camera tracking and a geometry-aware Gaussian surfel mapping module, introducing an information filter-based fusion method that explicitly accounts for sensor noise to achieve high-precision surface reconstruction. The proposed differentiable Gaussian surfel mapping effectively models multi-view consistent surfaces while enabling efficient parameter optimization. Extensive experimental results demonstrate that the proposed system achieves a surface reconstruction error of 0.6 extit{cm} on standardized benchmark datasets including Replica and ScanNet++, representing over 20% improvement in accuracy compared to state-of-the-art (SOTA) GS-based methods. Notably, the system maintains real-time processing capabilities at 24 FPS, establishing it as one of the most accurate differentiable-rendering-based real-time reconstruction systems. Project Page: https://zju3dv.github.io/eggfusion/

Problem

• Addresses real-time 3D reconstruction challenges in differentiable rendering
• Improves geometric fidelity by handling sensor noise for precise surfaces
• Enables efficient optimization while maintaining real-time processing capabilities

Innovation

• Geometry-aware Gaussian surfel mapping for surface modeling
• Information filter-based fusion to handle sensor noise
• Real-time differentiable rendering at 24 FPS