CottonSim: Development of an autonomous visual-guided robotic cotton-picking system in the Gazebo

2025-05-08

AI Summary

To address autonomous cotton harvesting in agricultural fields, this paper introduces the first end-to-end Gazebo-based simulation benchmark tailored for agricultural harvesting. Built upon the Husky mobile robot platform and our custom Cotton-Eye vision perception system, the framework integrates RGB-D sensing, YOLOv8n-seg instance segmentation (mAP: 85.2%), SLAM-based mapping, and GPS/IMU sensor fusion for robust localization. It enables dual-mode navigation—map-based and GPS-guided—within a custom ROS-based virtual cotton field environment. The proposed method achieves a closed-loop pipeline for vision-guided autonomous navigation, real-time cotton detection, localization, and harvesting. Experimental results show a 96.7% success rate for map-based navigation (position error < 0.25 m) and 100% success for GPS-based navigation (angular error < 5×10⁻⁶°). All code, trained models, and the virtual environment are publicly released, establishing a reproducible, extensible simulation baseline for agricultural robotics algorithm development and evaluation.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Computer Vision: Vision for Robotics & Autonomous Driving
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Agentic search

Abstract

In this study, an autonomous visual-guided robotic cotton-picking system, built on a Clearpath's Husky robot platform and the Cotton-Eye perception system, was developed in the Gazebo robotic simulator. Furthermore, a virtual cotton farm was designed and developed as a Robot Operating System (ROS 1) package to deploy the robotic cotton picker in the Gazebo environment for simulating autonomous field navigation. The navigation was assisted by the map coordinates and an RGB-depth camera, while the ROS navigation algorithm utilized a trained YOLOv8n-seg model for instance segmentation. The model achieved a desired mean Average Precision (mAP) of 85.2%, a recall of 88.9%, and a precision of 93.0% for scene segmentation. The developed ROS navigation packages enabled our robotic cotton-picking system to autonomously navigate through the cotton field using map-based and GPS-based approaches, visually aided by a deep learning-based perception system. The GPS-based navigation approach achieved a 100% completion rate (CR) with a threshold of 5 x 10^-6 degrees, while the map-based navigation approach attained a 96.7% CR with a threshold of 0.25 m. This study establishes a fundamental baseline of simulation for future agricultural robotics and autonomous vehicles in cotton farming and beyond. CottonSim code and data are released to the research community via GitHub: https://github.com/imtheva/CottonSim

Problem

• Develop autonomous visual-guided robotic cotton-picking system
• Simulate cotton farm navigation using ROS and Gazebo
• Achieve high-precision segmentation for field navigation

Innovation

• Autonomous visual-guided robotic cotton-picking system
• ROS navigation with YOLOv8n-seg model
• Simulated cotton farm in Gazebo environment