MeshSplat: Generalizable Sparse-View Surface Reconstruction via Gaussian Splatting

2025-08-25

AI Summary

Existing surface reconstruction methods struggle to accurately recover geometry from extremely sparse input views. To address this, we propose MeshSplat—a novel framework that pioneers the use of 2D Gaussian splats as a geometric representation, enabling generalizable, 3D-ground-truth-free surface reconstruction. Our method employs a feed-forward network to predict pixel-aligned 2D Gaussian distributions; integrates depth-map regularization and a normal alignment network to transfer novel-view synthesis capability to geometric reconstruction; and introduces a weighted Chamfer distance loss jointly optimized with a monocular normal estimator to guide normal prediction. Evaluated on sparse-view mesh reconstruction, MeshSplat achieves state-of-the-art performance, significantly improving geometric fidelity in complex scenes and enhancing cross-scene generalization.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Representation Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Surface reconstruction has been widely studied in computer vision and graphics. However, existing surface reconstruction works struggle to recover accurate scene geometry when the input views are extremely sparse. To address this issue, we propose MeshSplat, a generalizable sparse-view surface reconstruction framework via Gaussian Splatting. Our key idea is to leverage 2DGS as a bridge, which connects novel view synthesis to learned geometric priors and then transfers these priors to achieve surface reconstruction. Specifically, we incorporate a feed-forward network to predict per-view pixel-aligned 2DGS, which enables the network to synthesize novel view images and thus eliminates the need for direct 3D ground-truth supervision. To improve the accuracy of 2DGS position and orientation prediction, we propose a Weighted Chamfer Distance Loss to regularize the depth maps, especially in overlapping areas of input views, and also a normal prediction network to align the orientation of 2DGS with normal vectors predicted by a monocular normal estimator. Extensive experiments validate the effectiveness of our proposed improvement, demonstrating that our method achieves state-of-the-art performance in generalizable sparse-view mesh reconstruction tasks. Project Page: https://hanzhichang.github.io/meshsplat_web

Problem

• Reconstructs accurate surfaces from sparse input views
• Leverages Gaussian Splatting for novel view synthesis
• Improves geometry prediction via depth and normal regularization

Innovation

• Leverages 2DGS to connect view synthesis with geometric priors
• Uses Weighted Chamfer Distance Loss for depth regularization
• Employs normal prediction network to align 2DGS orientation