Advancement and Field Evaluation of a Dual-arm Apple Harvesting Robot

2025-06-06

AI Summary

To address the labor-intensive, costly, and hazardous nature of manual apple harvesting—and the poor adaptability and low reliability of existing robotic systems in complex orchard environments—this paper proposes a dual-arm harvesting robot designed for real-world orchards. Methodologically, we introduce a dynamic vacuum allocation mechanism and a dual-arm cooperative fault-tolerant strategy, integrated with a multi-degree-of-freedom adjustable platform. For robust fruit localization, we propose a cascaded approach combining foundation models, semantic segmentation, and depth-based clustering, fusing Time-of-Flight (ToF) vision with tactile pressure feedback. Field evaluations across two commercial orchards in Michigan achieved harvesting success rates of 80.7% and 79.7%, respectively, with an average per-fruit execution time of 5.97 seconds—28% faster than a single-arm baseline. The system demonstrates significantly enhanced operational efficiency and robustness in unstructured, cluttered orchard settings.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Vision for Robotics & Autonomous Driving
• Humans and AI: Planning and Decision Support for Human-Machine Teams

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Apples are among the most widely consumed fruits worldwide. Currently, apple harvesting fully relies on manual labor, which is costly, drudging, and hazardous to workers. Hence, robotic harvesting has attracted increasing attention in recent years. However, existing systems still fall short in terms of performance, effectiveness, and reliability for complex orchard environments. In this work, we present the development and evaluation of a dual-arm harvesting robot. The system integrates a ToF camera, two 4DOF robotic arms, a centralized vacuum system, and a post-harvest handling module. During harvesting, suction force is dynamically assigned to either arm via the vacuum system, enabling efficient apple detachment while reducing power consumption and noise. Compared to our previous design, we incorporated a platform movement mechanism that enables both in-out and up-down adjustments, enhancing the robot's dexterity and adaptability to varying canopy structures. On the algorithmic side, we developed a robust apple localization pipeline that combines a foundation-model-based detector, segmentation, and clustering-based depth estimation, which improves performance in orchards. Additionally, pressure sensors were integrated into the system, and a novel dual-arm coordination strategy was introduced to respond to harvest failures based on sensor feedback, further improving picking efficiency. Field demos were conducted in two commercial orchards in MI, USA, with different canopy structures. The system achieved success rates of 0.807 and 0.797, with an average picking cycle time of 5.97s. The proposed strategy reduced harvest time by 28% compared to a single-arm baseline. The dual-arm harvesting robot enhances the reliability and efficiency of apple picking. With further advancements, the system holds strong potential for autonomous operation and commercialization for the apple industry.

Problem

• Developing a dual-arm robot for efficient apple harvesting
• Improving performance in complex orchard environments
• Reducing manual labor costs and hazards

Innovation

• Dual-arm robot with dynamic vacuum suction
• Platform movement for canopy adaptability
• Robust apple localization using foundation models