PLANesT-3D: A new annotated dataset for segmentation of 3D plant point clouds

2024-07-30

arXiv.org

AI Summary

High-quality, diverse, publicly available annotated 3D point cloud datasets for organ-level plant segmentation are scarce. Method: We introduce PLANesT-3D—the first benchmark dataset for semantic and instance segmentation of colored plant point clouds—comprising 34 specimens of chili pepper, miniature rose, and red currant, with fine-grained “leaf/stem” semantic labels and organ-level instance annotations. We further propose SP-LSCnet, an end-to-end deep network integrating unsupervised superpoint extraction (SP) and LSC feature learning, built upon a PointNet++ backbone enhanced with RoseSegNet-inspired architectural optimizations. Contribution/Results: Experiments on PLANesT-3D demonstrate that SP-LSCnet significantly outperforms both PointNet++ and RoseSegNet. This work establishes the first fine-grained plant point cloud segmentation benchmark and delivers a lightweight, transferable segmentation model, thereby laying foundational data and algorithmic groundwork for intelligent agricultural perception.

Technology Category

• Computer Vision: Segmentation
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Creation of new annotated public datasets is crucial in helping advances in 3D computer vision and machine learning meet their full potential for automatic interpretation of 3D plant models. In this paper, we introduce PLANesT-3D; a new annotated dataset of 3D color point clouds of plants. PLANesT-3D is composed of 34 point cloud models representing 34 real plants from three different plant species: extit{Capsicum annuum}, extit{Rosa kordana}, and extit{Ribes rubrum}. Both semantic labels in terms of"leaf"and"stem", and organ instance labels were manually annotated for the full point clouds. As an additional contribution, SP-LSCnet, a novel semantic segmentation method that is a combination of unsupervised superpoint extraction and a 3D point-based deep learning approach is introduced and evaluated on the new dataset. Two existing deep neural network architectures, PointNet++ and RoseSegNet were also tested on the point clouds of PLANesT-3D for semantic segmentation.

Problem

• Introduces PLANesT-3D dataset for 3D plant point cloud segmentation
• Addresses lack of diverse species and acquisition modalities in existing datasets
• Evaluates new SP-LSCnet method for semantic segmentation on plant models

Innovation

• Introduces PLANesT-3D dataset with 3D plant point clouds
• Uses low-cost SfM/MVS technique for data acquisition
• Proposes SP-LSCnet method combining superpoint extraction and deep learning