🤖 AI Summary
To address inaccurate fruit harvesting-point localization—leading to fruit damage and drop—in agricultural harvesting robots, this paper proposes a dual-path localization method integrating multi-view RGB-D vision with model-driven reasoning. Innovatively, motion-capture calibration is fused with surface points from two calibrated cameras (Cfix/Ceih) to construct a multi-view geometric model; furthermore, AdaBoost regression is first introduced and empirically validated for harvesting-point prediction, demonstrating superior robustness over conventional single-camera analytical methods. Experiments show that the proposed method achieves an 88.8% harvesting success rate (median localization error: 4.40 mm), outperforming the single-camera baseline by 11.1 percentage points and significantly improving overall robotic harvesting performance. Key contributions include: (i) the first multi-view geometric modeling framework tailored for fruit harvesting-point localization, and (ii) the first application and validation of AdaBoost regression for this task, establishing its superiority in accuracy and robustness.
📝 Abstract
This paper presents multi-vision-based localisation strategies for harvesting robots. Identifying picking points accurately is essential for robotic harvesting because insecure grasping can lead to economic loss through fruit damage and dropping. In this study, two multi-vision-based localisation methods, namely the analytical approach and model-based algorithms, were employed. The actual geometric centre points of fruits were collected using a motion capture system (mocap), and two different surface points Cfix and Ceih were extracted using two Red-Green-Blue-Depth (RGB-D) cameras. First, the picking points of the target fruit were detected using analytical methods. Second, various primary and ensemble learning methods were employed to predict the geometric centre of target fruits by taking surface points as input. Adaboost regression, the most successful model-based localisation algorithm, achieved 88.8% harvesting accuracy with a Mean Euclidean Distance (MED) of 4.40 mm, while the analytical approach reached 81.4% picking success with a MED of 14.25 mm, both demonstrating better performance than the single-camera, which had a picking success rate of 77.7% with a MED of 24.02 mm. To evaluate the effect of picking point accuracy in collecting fruits, a series of robotic harvesting experiments were performed utilising a collaborative robot (cobot). It is shown that multi-vision systems can improve picking point localisation, resulting in higher success rates of picking in robotic harvesting.