OMeGa: Joint Optimization of Explicit Meshes and Gaussian Splats for Robust Scene-Level Surface Reconstruction

2025-09-29

AI Summary

To address inaccurate geometric reconstruction in textureless indoor regions and the decoupling between mesh extraction and Gaussian point optimization, this paper proposes a novel end-to-end framework for jointly optimizing an explicit triangular mesh and 2D Gaussians (splats). A differentiable binding mechanism dynamically links Gaussian spatial attributes to mesh vertices, while an iterative, geometry-error-driven mesh refinement strategy is introduced. Geometric regularization is enhanced by integrating monocular normal supervision with explicit mesh constraints; robustness is further improved via adaptive patch partitioning and pruning. This work presents the first differentiable co-optimization of meshes and Gaussians. On mainstream indoor reconstruction benchmarks, it achieves a 47.3% reduction in Chamfer-L1 distance over 2DGS—setting a new state-of-the-art—while maintaining high-fidelity novel-view synthesis.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Computer Vision: Diffusion Models for Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Neural rendering with Gaussian splatting has advanced novel view synthesis, and most methods reconstruct surfaces via post-hoc mesh extraction. However, existing methods suffer from two limitations: (i) inaccurate geometry in texture-less indoor regions, and (ii) the decoupling of mesh extraction from optimization, thereby missing the opportunity to leverage mesh geometry to guide splat optimization. In this paper, we present OMeGa, an end-to-end framework that jointly optimizes an explicit triangle mesh and 2D Gaussian splats via a flexible binding strategy, where spatial attributes of Gaussian Splats are expressed in the mesh frame and texture attributes are retained on splats. To further improve reconstruction accuracy, we integrate mesh constraints and monocular normal supervision into the optimization, thereby regularizing geometry learning. In addition, we propose a heuristic, iterative mesh-refinement strategy that splits high-error faces and prunes unreliable ones to further improve the detail and accuracy of the reconstructed mesh. OMeGa achieves state-of-the-art performance on challenging indoor reconstruction benchmarks, reducing Chamfer-$L_1$ by 47.3% over the 2DGS baseline while maintaining competitive novel-view rendering quality. The experimental results demonstrate that OMeGa effectively addresses prior limitations in indoor texture-less reconstruction.

Problem

• Jointly optimizes explicit meshes and Gaussian splats for reconstruction
• Improves geometry accuracy in texture-less indoor regions
• Integrates mesh constraints and normal supervision for regularization

Innovation

• Jointly optimizes explicit mesh and Gaussian splats
• Integrates mesh constraints and normal supervision
• Uses iterative mesh-refinement strategy for accuracy