NeuraLeaf: Neural Parametric Leaf Models with Shape and Deformation Disentanglement

2025-07-16

AI Summary

Accurate 3D reconstruction of plant leaves remains challenging due to their high morphological diversity and significant deformability. Method: We propose a neural parametric model that decouples 2D base shape representation from differentiable 3D deformation. Specifically, we employ a neural implicit field to encode leaf geometry and introduce a skeleton-free skinning mechanism for flexible deformation modeling. Furthermore, we design a joint 2D image–3D geometry optimization framework and introduce DeformLeaf—the first large-scale, 3D deformable leaf dataset tailored for deformation-aware modeling. Contribution/Results: Our method achieves the first unified parametric representation across diverse plant species. It enables high-fidelity geometric fitting from sparse observations (e.g., depth maps or point clouds), ensures precise texture alignment, and delivers robust reconstructions under complex poses. Quantitative and qualitative evaluations demonstrate substantial improvements in both morphological diversity coverage and reconstruction accuracy over state-of-the-art approaches.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Learning with Manifolds
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We develop a neural parametric model for 3D leaves for plant modeling and reconstruction that are essential for agriculture and computer graphics. While neural parametric models are actively studied for humans and animals, plant leaves present unique challenges due to their diverse shapes and flexible deformation. To this problem, we introduce a neural parametric model for leaves, NeuraLeaf. Capitalizing on the fact that flattened leaf shapes can be approximated as a 2D plane, NeuraLeaf disentangles the leaves' geometry into their 2D base shapes and 3D deformations. This representation allows learning from rich sources of 2D leaf image datasets for the base shapes, and also has the advantage of simultaneously learning textures aligned with the geometry. To model the 3D deformation, we propose a novel skeleton-free skinning model and create a newly captured 3D leaf dataset called DeformLeaf. We show that NeuraLeaf successfully generates a wide range of leaf shapes with deformation, resulting in accurate model fitting to 3D observations like depth maps and point clouds. Our implementation and dataset are available at https://neuraleaf-yang.github.io/.

Problem

• Modeling 3D leaves for agriculture and graphics
• Disentangling leaf shape and deformation challenges
• Creating accurate leaf models from 2D/3D data

Innovation

• Neural parametric model for 3D leaves
• Disentangles 2D base shapes and 3D deformations
• Novel skeleton-free skinning model for deformation