ST-DETrack: Identity-Preserving Branch Tracking in Entangled Plant Canopies via Dual Spatiotemporal Evidence

2025-12-17

AI Summary

This study addresses the challenge of long-term, identity-consistent tracking of individual plant stems across the entire phenological continuum—from bud emergence to flowering—within densely entangled canopies, where non-rigid growth and severe identity fragmentation impede robust tracking. We propose a spatiotemporal dual-decoder architecture that jointly integrates adaptive feature gating, optical-flow-guided temporal modeling, and geometric-angular joint representation. Crucially, we introduce a novel physics-informed mechanism embedding a biologically grounded negative gravitropism constraint, enabling dynamic balancing between geometric priors and motion consistency. Evaluated on a *Brassica napus* (rapeseed) dataset, our method achieves 93.6% stem matching accuracy—outperforming pure spatial and pure temporal baselines by 28.9% and 3.3%, respectively. The framework significantly enhances long-term identity robustness and biological interpretability, advancing automated, morphology-aware phenotyping in complex plant architectures.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Low Level & Physics-based Vision
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Automated extraction of individual plant branches from time-series imagery is essential for high-throughput phenotyping, yet it remains computationally challenging due to non-rigid growth dynamics and severe identity fragmentation within entangled canopies. To overcome these stage-dependent ambiguities, we propose ST-DETrack, a spatiotemporal-fusion dual-decoder network designed to preserve branch identity from budding to flowering. Our architecture integrates a spatial decoder, which leverages geometric priors such as position and angle for early-stage tracking, with a temporal decoder that exploits motion consistency to resolve late-stage occlusions. Crucially, an adaptive gating mechanism dynamically shifts reliance between these spatial and temporal cues, while a biological constraint based on negative gravitropism mitigates vertical growth ambiguities. Validated on a Brassica napus dataset, ST-DETrack achieves a Branch Matching Accuracy (BMA) of 93.6%, significantly outperforming spatial and temporal baselines by 28.9 and 3.3 percentage points, respectively. These results demonstrate the method's robustness in maintaining long-term identity consistency amidst complex, dynamic plant architectures.

Problem

• Automated extraction of individual plant branches from time-series imagery
• Preserving branch identity amidst non-rigid growth and severe occlusion
• Resolving identity fragmentation in entangled plant canopies over time

Innovation

• Spatiotemporal fusion dual-decoder network for branch tracking
• Adaptive gating mechanism balances spatial and temporal cues
• Biological constraint based on negative gravitropism resolves ambiguities