Exploiting Boundary Loss for the Hierarchical Panoptic Segmentation of Plants and Leaves

2024-12-31

AI Summary

To address the fine-grained identification of leaves and weeds in precision agriculture, this paper proposes an end-to-end hierarchical panoptic segmentation framework that jointly performs pixel-level classification, instance-level counting, and localization for plants, leaves, and weeds. We innovatively integrate boundary-aware loss into the Mask2Former architecture, jointly optimized with focal loss to enhance segmentation boundary accuracy and instance separation—particularly for small, densely overlapping leaves and slender weeds. On standard benchmarks, our method achieves a panoptic quality (PQ+) score of 81.89, reduces mean absolute error in leaf counting by 37.2%, and significantly improves the accuracy of targeted fertilization and weeding decisions. The source code is publicly available.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Learning to Search
• Machine Learning: Hardware-aware ML

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Precision agriculture leverages data and machine learning so that farmers can monitor their crops and target interventions precisely. This enables the precision application of herbicide only to weeds, or the precision application of fertilizer only to undernourished crops, rather than to the entire field. The approach promises to maximize yields while minimizing resource use and harm to the surrounding environment. To this end, we propose a hierarchical panoptic segmentation method that simultaneously determines leaf count (as an identifier of plant growth)and locates weeds within an image. In particular, our approach aims to improve the segmentation of smaller instances like the leaves and weeds by incorporating focal loss and boundary loss. Not only does this result in competitive performance, achieving a PQ+ of 81.89 on the standard training set, but we also demonstrate we can improve leaf-counting accuracy with our method. The code is available at https://github.com/madeleinedarbyshire/HierarchicalMask2Former.

Problem

• Precision Agriculture
• Plant Weed Discrimination
• Resource Efficiency

Innovation

• Boundary Loss
• Plant Leaf Detection
• Weed Discrimination