LeafFit: Plant Assets Creation from 3D Gaussian Splatting

2026-02-12

AI Summary

This work proposes a novel method to convert high-fidelity 3D Gaussian Splatting (3DGS) representations of plants into editable, lightweight instanced meshes. Leveraging the repetitive structural nature of vegetation, the approach introduces a repetition prior into the 3DGS-to-mesh conversion pipeline for the first time: individual leaves are extracted via interactive-assisted segmentation to construct a representative template, which is then fitted to all instances using differentiable moving least squares (MLS) deformation. The resulting representation supports efficient runtime deformation via vertex shaders, achieving high geometric fidelity while significantly reducing storage overhead and enabling parametric editing. Compared to existing techniques, the method demonstrates notable improvements in segmentation quality, deformation accuracy, and real-time performance, making it well-suited for integration into game engines and real-time graphics production pipelines.

Technology Category

• Computer Vision: 3D Computer Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Machine Learning: Representation Learning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

We propose LeafFit, a pipeline that converts 3D Gaussian Splatting (3DGS) of individual plants into editable, instanced mesh assets. While 3DGS faithfully captures complex foliage, its high memory footprint and lack of mesh topology make it incompatible with traditional game production workflows. We address this by leveraging the repetition of leaf shapes; our method segments leaves from the unstructured 3DGS, with optional user interaction included as a fallback. A representative leaf group is selected and converted into a thin, sharp mesh to serve as a template; this template is then fitted to all other leaves via differentiable Moving Least Squares (MLS) deformation. At runtime, the deformation is evaluated efficiently on-the-fly using a vertex shader to minimize storage requirements. Experiments demonstrate that LeafFit achieves higher segmentation quality and deformation accuracy than recent baselines while significantly reducing data size and enabling parameter-level editing.

Problem

• 3D Gaussian Splatting
• mesh assets
• foliage modeling
• game production workflows
• editable plant representation

Innovation

• 3D Gaussian Splatting
• leaf segmentation
• differentiable MLS deformation
• mesh asset creation
• real-time vertex shader deformation