GS-NBV: a Geometry-based, Semantics-aware Viewpoint Planning Algorithm for Avocado Harvesting under Occlusions

2025-06-29

AI Summary

To address the challenge of harvesting-point localization in avocado picking—caused by irregular fruit morphology, severe occlusion, and unstructured environments—this paper proposes a geometry- and semantics-aware closed-loop viewpoint planning method. The approach comprises three stages: (1) spatial sampling guided by a novel 1D circular constraint; (2) multimodal viewpoint evaluation integrating object detection, 3D geometric reconstruction, and semantic analysis; and (3) score-driven selection and execution of the optimal viewpoint. A key innovation is the 1D circular constraint, which drastically reduces the viewpoint search space, alongside a new harvesting-score metric jointly optimizing visibility, pose stability, and robotic reachability. Evaluated in two highly occluded simulated scenarios, the method achieves 100% harvesting success rate, substantially outperforming state-of-the-art algorithms. The implementation is publicly available.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Vision for Robotics & Autonomous Driving

Application Category

• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Efficient identification of picking points is critical for automated fruit harvesting. Avocados present unique challenges owing to their irregular shape, weight, and less-structured growing environments, which require specific viewpoints for successful harvesting. We propose a geometry-based, semantics-aware viewpoint-planning algorithm to address these challenges. The planning process involves three key steps: viewpoint sampling, evaluation, and execution. Starting from a partially occluded view, the system first detects the fruit, then leverages geometric information to constrain the viewpoint search space to a 1D circle, and uniformly samples four points to balance the efficiency and exploration. A new picking score metric is introduced to evaluate the viewpoint suitability and guide the camera to the next-best view. We validate our method through simulation against two state-of-the-art algorithms. Results show a 100% success rate in two case studies with significant occlusions, demonstrating the efficiency and robustness of our approach. Our code is available at https://github.com/lineojcd/GSNBV

Problem

• Efficiently identify avocado picking points under occlusions
• Address irregular shape and unstructured growing environments
• Develop viewpoint planning for successful harvesting

Innovation

• Geometry-based viewpoint planning for occlusions
• Semantics-aware picking score metric
• 1D circle constrained viewpoint search