WeedSense: Multi-Task Learning for Weed Segmentation, Height Estimation, and Growth Stage Classification

2025-08-20

AI Summary

Weed management remains a critical bottleneck for sustainable agriculture. This paper proposes a lightweight multi-task learning framework that jointly performs pixel-level weed segmentation, height estimation, and growth-stage classification in agricultural fields. Our method introduces a novel dual-path encoder and a branched decoder architecture, integrating inverted bottleneck modules with Transformer-based cross-scale feature interaction to enable synergistic optimization of all three tasks within a single model. Evaluated on a custom-built 16-class weed dataset, the framework achieves 89.78% mIoU for segmentation, 1.67 cm mean absolute error for height estimation, and 99.99% accuracy for growth-stage classification, while running at 160 FPS. Compared to cascaded single-task models, it reduces inference latency by 3× and significantly lowers parameter count and computational cost. The proposed approach establishes a new paradigm for precise, real-time, and resource-efficient intelligent weed management.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Learning to Search
• Machine Learning: Hardware-aware ML

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Web Mining and Content Analysis: Large pretrained models with web data
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Weed management represents a critical challenge in agriculture, significantly impacting crop yields and requiring substantial resources for control. Effective weed monitoring and analysis strategies are crucial for implementing sustainable agricultural practices and site-specific management approaches. We introduce WeedSense, a novel multi-task learning architecture for comprehensive weed analysis that jointly performs semantic segmentation, height estimation, and growth stage classification. We present a unique dataset capturing 16 weed species over an 11-week growth cycle with pixel-level annotations, height measurements, and temporal labels. WeedSense leverages a dual-path encoder incorporating Universal Inverted Bottleneck blocks and a Multi-Task Bifurcated Decoder with transformer-based feature fusion to generate multi-scale features and enable simultaneous prediction across multiple tasks. WeedSense outperforms other state-of-the-art models on our comprehensive evaluation. On our multi-task dataset, WeedSense achieves mIoU of 89.78% for segmentation, 1.67cm MAE for height estimation, and 99.99% accuracy for growth stage classification while maintaining real-time inference at 160 FPS. Our multitask approach achieves 3$ imes$ faster inference than sequential single-task execution and uses 32.4% fewer parameters. Please see our project page at weedsense.github.io.

Problem

• Simultaneously performs weed segmentation, height estimation, and growth stage classification
• Addresses comprehensive weed analysis for sustainable agricultural management practices
• Develops multi-task learning architecture for real-time weed monitoring applications

Innovation

• Multi-task learning for weed analysis
• Dual-path encoder with Universal Inverted Bottleneck blocks
• Transformer-based feature fusion in decoder