Tessellation GS: Neural Mesh Gaussians for Robust Monocular Reconstruction of Dynamic Objects

2025-12-08

AI Summary

To address the challenges of anisotropic Gaussian splatting (GS) in monocular dynamic scene reconstruction—including poor view extrapolation, limited generalization under sparse-view conditions, and overfitting—this paper proposes a mesh-anchored, structured 2D Gaussian splatting method. Specifically, Gaussian distributions are constrained to deformable triangular mesh surfaces; neural implicit features are hierarchically modeled, and mesh deformation is initialized using geometry priors from foundation models. Adaptive patch subdivision and detail-aware loss functions further drive optimization. Evaluated on dynamic object reconstruction, our approach significantly improves fidelity and generalization: LPIPS decreases by 29.1% and Chamfer distance reduces by 49.2% compared to prior state-of-the-art methods.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Learning with Manifolds
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

3D Gaussian Splatting (GS) enables highly photorealistic scene reconstruction from posed image sequences but struggles with viewpoint extrapolation due to its anisotropic nature, leading to overfitting and poor generalization, particularly in sparse-view and dynamic scene reconstruction. We propose Tessellation GS, a structured 2D GS approach anchored on mesh faces, to reconstruct dynamic scenes from a single continuously moving or static camera. Our method constrains 2D Gaussians to localized regions and infers their attributes via hierarchical neural features on mesh faces. Gaussian subdivision is guided by an adaptive face subdivision strategy driven by a detail-aware loss function. Additionally, we leverage priors from a reconstruction foundation model to initialize Gaussian deformations, enabling robust reconstruction of general dynamic objects from a single static camera, previously extremely challenging for optimization-based methods. Our method outperforms previous SOTA method, reducing LPIPS by 29.1% and Chamfer distance by 49.2% on appearance and mesh reconstruction tasks.

Problem

• Overcomes anisotropic limitations in 3D Gaussian Splatting for dynamic scenes
• Enables robust monocular reconstruction of dynamic objects from sparse views
• Improves generalization via mesh-anchored Gaussians with adaptive subdivision strategy

Innovation

• Anchors 2D Gaussians on mesh faces for structure
• Uses adaptive face subdivision guided by detail-aware loss
• Leverages foundation model priors to initialize Gaussian deformations