Assessing SAM for Tree Crown Instance Segmentation from Drone Imagery

2025-03-26

AI Summary

Traditional crown segmentation methods for afforestation monitoring are costly, time-consuming, and heavily reliant on large-scale annotated data. Method: This study systematically evaluates the applicability of the Segment Anything Model (SAM) to individual-tree crown instance segmentation in high-resolution UAV remote sensing imagery. We propose a multi-source prompting strategy integrating Digital Surface Models (DSM) and explore SAM fine-tuning under low-shot annotation settings. Contribution/Results: While the zero-shot SAM underperforms a customized Mask R-CNN baseline, incorporating DSM-based prompts improves IoU by 12.6%; subsequent fine-tuning enables SAM to surpass the baseline. This work is the first to empirically validate SAM’s transferability and optimization pathways for tree crown segmentation under data-scarce conditions. It establishes a lightweight, scalable methodology for few-shot remote sensing interpretation—offering a practical alternative to labor-intensive annotation pipelines in forestry monitoring.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Ensemble Methods
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

The potential of tree planting as a natural climate solution is often undermined by inadequate monitoring of tree planting projects. Current monitoring methods involve measuring trees by hand for each species, requiring extensive cost, time, and labour. Advances in drone remote sensing and computer vision offer great potential for mapping and characterizing trees from aerial imagery, and large pre-trained vision models, such as the Segment Anything Model (SAM), may be a particularly compelling choice given limited labeled data. In this work, we compare SAM methods for the task of automatic tree crown instance segmentation in high resolution drone imagery of young tree plantations. We explore the potential of SAM for this task, and find that methods using SAM out-of-the-box do not outperform a custom Mask R-CNN, even with well-designed prompts, but that there is potential for methods which tune SAM further. We also show that predictions can be improved by adding Digital Surface Model (DSM) information as an input.

Problem

• Assessing SAM for tree crown segmentation in drone imagery
• Comparing SAM methods for automatic tree crown instance segmentation
• Improving predictions with Digital Surface Model (DSM) input

Innovation

• Uses SAM for tree crown segmentation
• Compares SAM with Mask R-CNN
• Enhances predictions with DSM data