DMesh: A Differentiable Mesh Representation

2024-04-20

Neural Information Processing Systems

AI Summary

Existing explicit 3D triangular mesh representations lack simultaneous topological flexibility and full differentiability. Method: We propose DMesh, a differentiable triangular mesh representation that jointly models geometry and connectivity. DMesh generates candidate faces via weighted Delaunay tetrahedralization and models face existence probabilistically through a differentiable parametric model, enabling arbitrary topology. Contribution/Results: DMesh enables the first end-to-end differentiable explicit mesh reconstruction, overcoming the dual limitations of traditional explicit methods (fixed topology) and implicit methods (non-differentiable or topologically ambiguous). Evaluated on point cloud and multi-view image inputs, DMesh achieves topology-adaptive, high-fidelity, and optimization-stable mesh reconstruction, significantly improving geometric accuracy and training robustness compared to prior approaches.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning with Manifolds
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

We present a differentiable representation, DMesh, for general 3D triangular meshes. DMesh considers both the geometry and connectivity information of a mesh. In our design, we first get a set of convex tetrahedra that compactly tessellates the domain based on Weighted Delaunay Triangulation (WDT), and select triangular faces on the tetrahedra to define the final mesh. We formulate probability of faces to exist on the actual surface in a differentiable manner based on the WDT. This enables DMesh to represent meshes of various topology in a differentiable way, and allows us to reconstruct the mesh under various observations, such as point cloud and multi-view images using gradient-based optimization. The source code and full paper is available at: https://sonsang.github.io/dmesh-project.

Problem

• Develops DMesh for differentiable 3D triangular mesh representation
• Combines geometry and connectivity via Weighted Delaunay Triangulation
• Enables gradient-based mesh reconstruction from observations

Innovation

• Differentiable mesh representation using convex tetrahedra
• Probability-based surface face selection via WDT
• Gradient optimization for mesh reconstruction