Experimental Study on Automatically Assembling Custom Catering Packages With a 3-DOF Delta Robot Using Deep Learning Methods

2024-05-14

2024 32nd International Conference on Electrical Engineering (ICEE)

AI Summary

This study addresses the need for fully automated sorting and boxing of Persian culinary items using a 3-DOF Delta parallel robot. To overcome the lack of domain-specific visual data, we construct the first Persian food product dataset comprising 1,500 annotated images. We propose a model-free, vision-guided grasping method that eliminates reliance on object CAD models or pose priors: it jointly leverages segmentation masks and principal component analysis to fit oriented bounding rectangles and geometrically solve for dual grasp points. The system integrates YOLOv5 for detection, FastSAM for instance segmentation, mask-driven pose estimation, covariance-based eigenvector computation for grasp point selection, and ROS-enabled real-time motion control. Experiments demonstrate an end-to-end grasping success rate exceeding 80%, enabling closed-loop autonomous operation spanning detection, localization, pose estimation, grasp planning, and execution.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Vision for Robotics & Autonomous Driving
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This paper introduces a pioneering experimental study on the automated packing of a catering package using a two-fingered gripper affixed to a 3-degree-of-freedom Delta parallel robot. A distinctive contribution lies in the application of a deep learning approach to tackle this challenge. A custom dataset, comprising 1,500 images, is meticulously curated for this endeavor, representing a noteworthy initiative as the first dataset focusing on Persian-manufactured products. The study employs the YOLOV5 model for object detection, followed by segmentation using the FastSAM model. Subsequently, rotation angle calculation is facilitated with segmentation masks, and a rotated rectangle encapsulating the object is generated. This rectangle forms the basis for calculating two grasp points using a novel geometrical approach involving eigenvectors. An extensive experimental study validates the proposed model, where all pertinent information is seamlessly transmitted to the 3-DOF Delta parallel robot. The proposed algorithm ensures real-time detection, calibration, and the fully autonomous packing process of a catering package, boasting an impressive over 80% success rate in automatic grasping. This study marks a significant stride in advancing the capabilities of robotic systems for practical applications in packaging automation.

Problem

• Automating catering package assembly using a 3-DOF Delta robot
• Applying deep learning for object detection and segmentation
• Achieving real-time autonomous packing with high success rate

Innovation

• 3-DOF Delta robot with deep learning
• YOLOV5 and FastSAM for detection
• Geometric grasp points calculation