Zero-Shot 3D Plant Organ Segmentation with SAM3 and Semantic NeRFs

2026-09-07

AI Summary

本文提出了一种无需标注的3D植物器官分割方法，结合了文本提示的SAM3分割与语义NeRFs，仅使用多视角RGB图像和类别名称列表即可生成准确的3D标签。

Abstract

Accurate 3D plant organ segmentation is fundamental to automated phenotyping. Existing approaches rely on annotated training data or species-specific model configurations. We present an annotation-free pipeline for 3D plant organ segmentation, combining text-prompted SAM3 segmentation with semantic neural radiance fields (NeRFs). Given only multi-view RGB images and a list of class names, our zero-shot pipeline produces semantically labeled 3D point clouds without manual annotation, per-species fine-tuning, or domain-specific preprocessing. Multi-view NeRF fusion acts as effective implicit consensus mechanism that lifts imperfect per-frame masks into accurate 3D labels. On a controlled Begonia maculata testbed the SAM3 pipeline achieves 92.6% mIoU, reaching 95.9% of the oracle upper bound established with perfect ground-truth masks. The pipeline was further evaluated on a new dataset spanning ten diverse plant point clouds reaching an average 0.856 mIoU, with leaf and pot IoU above 0.91 and 0.90 for every species, respectively. These results demonstrate that annotation-free 3D plant organ segmentation is now feasible and approaching the range of supervised methods.

Problem

• 3D plant organ segmentation
• automated phenotyping
• annotation-free
• zero-shot

Innovation

• zero-shot
• annotation-free
• SAM3
• semantic NeRFs
• 3D plant organ segmentation