Safe Leaf Manipulation for Accurate Shape and Pose Estimation of Occluded Fruits

2024-09-25

arXiv.org

AI Summary

To address the challenge of accurate fruit shape and pose estimation by agricultural robots under severe occlusion, this paper proposes an active, safety-aware leaf-parting strategy. The method employs perception-driven deformable modeling to enable low-damage physical interaction with leaves, and integrates scene-consistent 3D shape completion to jointly recover occluded fruit geometry via visuo-tactile sensing. Crucially, it introduces the first joint optimization framework coupling leaf deformability modeling with fruit shape completion, overcoming the limitations of conventional passive perception. Experiments on both synthetic and real sweet pepper plants demonstrate that our approach reduces fruit pose estimation error by 32.7% and improves shape reconstruction IoU by 28.4% over state-of-the-art baselines, while maintaining leaf damage below 5%. This work establishes a novel paradigm for active perception and precise robotic manipulation in heavily occluded agricultural environments.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Low Level & Physics-based Vision
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Fruit monitoring plays an important role in crop management, and rising global fruit consumption combined with labor shortages necessitates automated monitoring with robots. However, occlusions from plant foliage often hinder accurate shape and pose estimation. Therefore, we propose an active fruit shape and pose estimation method that physically manipulates occluding leaves to reveal hidden fruits. This paper introduces a framework that plans robot actions to maximize visibility and minimize leaf damage. We developed a novel scene-consistent shape completion technique to improve fruit estimation under heavy occlusion and utilize a perception-driven deformation graph model to predict leaf deformation during planning. Experiments on artificial and real sweet pepper plants demonstrate that our method enables robots to safely move leaves aside, exposing fruits for accurate shape and pose estimation, outperforming baseline methods. Project page: https://shaoxiongyao.github.io/lmap-ssc/.

Problem

• Automated fruit monitoring with robots
• Accurate shape and pose estimation of occluded fruits
• Safe leaf manipulation to minimize damage

Innovation

• Active leaf manipulation technique
• Scene-consistent shape completion
• Perception-driven deformation graph model