DeMapGS: Simultaneous Mesh Deformation and Surface Attribute Mapping via Gaussian Splatting

2025-12-11

AI Summary

Existing Gaussian splatting methods struggle to model deformable surfaces and surface-attached attributes, suffering from topological inconsistency and limited editing flexibility. This paper proposes the first end-to-end framework that anchors 2D Gaussian ellipsoids onto a deformable template mesh, jointly optimizing geometric deformation and surface attribute mappings—including diffuse albedo, normals, and displacement. We introduce four key innovations: (i) a structured Gaussian parameterization enforcing spatial coherence; (ii) differentiable mesh deformation modeling; (iii) an alternating 2D/3D rendering mechanism for consistent supervision; and (iv) gradient diffusion-based supervision to significantly improve optimization robustness—especially in concave regions. Our method achieves state-of-the-art mesh reconstruction quality while enabling high-fidelity, parameter-sharing-based editing and cross-object attribute transfer—demonstrating strong generalization for downstream applications.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning with Manifolds
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We propose DeMapGS, a structured Gaussian Splatting framework that jointly optimizes deformable surfaces and surface-attached 2D Gaussian splats. By anchoring splats to a deformable template mesh, our method overcomes topological inconsistencies and enhances editing flexibility, addressing limitations of prior Gaussian Splatting methods that treat points independently. The unified representation in our method supports extraction of high-fidelity diffuse, normal, and displacement maps, enabling the reconstructed mesh to inherit the photorealistic rendering quality of Gaussian Splatting. To support robust optimization, we introduce a gradient diffusion strategy that propagates supervision across the surface, along with an alternating 2D/3D rendering scheme to handle concave regions. Experiments demonstrate that DeMapGS achieves state-of-the-art mesh reconstruction quality and enables downstream applications for Gaussian splats such as editing and cross-object manipulation through a shared parametric surface.

Problem

• Simultaneously deforms meshes and maps surface attributes
• Overcomes topological inconsistencies in Gaussian Splatting methods
• Enables high-fidelity texture extraction and photorealistic mesh rendering

Innovation

• Anchoring 2D Gaussian splats to deformable mesh template
• Using gradient diffusion strategy for robust surface optimization
• Employing alternating 2D/3D rendering scheme for concave regions