An RGB-D Image Dataset for Lychee Detection and Maturity Classification for Robotic Harvesting

2025-10-19

AI Summary

A high-quality, open-source RGB-D dataset for lychee detection and ripeness classification under natural field conditions is currently unavailable. Method: This study constructs the first RGB-D dataset tailored for robotic harvesting, encompassing 11,414 images and 9,658 finely annotated samples across multiple lychee cultivars, growth stages, and weather conditions. It innovatively fuses RGB and depth modalities, employs a rigorous annotation protocol—featuring independent multi-annotator labeling followed by expert verification—and applies image augmentation to enhance data diversity. Contribution/Results: Using this dataset, we systematically benchmark three state-of-the-art deep learning models on both detection and ripeness grading tasks. Experiments demonstrate substantial improvements in model generalization and discriminative accuracy. The dataset is publicly released to advance research in agricultural robotics and vision-based perception, effectively addressing a critical gap in open agricultural vision data.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Vision for Robotics & Autonomous Driving
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Lychee is a high-value subtropical fruit. The adoption of vision-based harvesting robots can significantly improve productivity while reduce reliance on labor. High-quality data are essential for developing such harvesting robots. However, there are currently no consistently and comprehensively annotated open-source lychee datasets featuring fruits in natural growing environments. To address this, we constructed a dataset to facilitate lychee detection and maturity classification. Color (RGB) images were acquired under diverse weather conditions, and at different times of the day, across multiple lychee varieties, such as Nuomici, Feizixiao, Heiye, and Huaizhi. The dataset encompasses three different ripeness stages and contains 11,414 images, consisting of 878 raw RGB images, 8,780 augmented RGB images, and 1,756 depth images. The images are annotated with 9,658 pairs of lables for lychee detection and maturity classification. To improve annotation consistency, three individuals independently labeled the data, and their results were then aggregated and verified by a fourth reviewer. Detailed statistical analyses were done to examine the dataset. Finally, we performed experiments using three representative deep learning models to evaluate the dataset. It is publicly available for academic

Problem

• Lack of annotated lychee datasets for robotic harvesting
• Need for lychee detection and maturity classification in natural environments
• Dataset addresses scarcity with RGB-D images across multiple varieties

Innovation

• RGB-D dataset for lychee detection and maturity classification
• Dataset includes 11,414 images with multi-stage ripeness annotations
• Three deep learning models evaluated on multi-variety lychee dataset