Still image and spatial-temporal tomato data enabling detection, segmentation, tracking, and video-instance segmentation using strong and weak labels

2026-07-16

AI Summary

Existing agricultural vision datasets struggle to support real-world tomato fruit detection, segmentation, tracking, and maturity estimation. To address this gap, this work introduces two high-quality tomato vision datasets tailored for commercial greenhouse environments: BUTom21, comprising static images with meticulously hand-annotated labels, and BUTom-ST21, consisting of video sequences annotated with AI-generated pseudo-labels; both provide pixel-level maturity annotations. By innovatively integrating strongly supervised and weakly supervised labeling strategies, this study achieves, for the first time, spatiotemporally coherent multi-task phenotyping data that enables not only single-frame instance segmentation but also cross-frame fruit tracking and dynamic maturity analysis. These datasets establish a realistic and challenging benchmark for agricultural computer vision, advancing field-based phenotyping technologies.

Abstract

In this manuscript we release two datasets for visual sensing of tomato plants grown in commercial-like settings and acquired using a robot. The first is BUTom21 which consists of still images and manual annotations. The second is BUTom-ST21 which consists of video-based data and semi-automated annotations through AI-based methods, referred to as pseudo-labels. In both cases, we provide pixel-level labels for the ripeness of the fruit. The aim is to provide the research community a challenging set of real-world imagery to explore methods to sense and estimate the state of tomato plants and their fruit, which is an important horticultural crop. Importantly, the spatial-temporal dataset provides individual fruit count and ripeness information enabling researchers to push the boundaries of field-based phenotyping.

Problem

• tomato phenotyping
• fruit detection
• ripeness estimation
• spatial-temporal data
• field-based sensing

Innovation

• pseudo-labels
• video-instance segmentation
• spatial-temporal dataset
• field-based phenotyping
• pixel-level annotation