SG-DOR: Learning Scene Graphs with Direction-Conditioned Occlusion Reasoning for Pepper Plants

2026-03-06

AI Summary

This work addresses the challenge of robotic harvesting in dense crop canopies, where effective intervention planning requires accurate identification of occluding plant organs and their directional dependencies relative to target fruits. The authors propose a direction-conditioned occlusion reasoning mechanism that, for the first time, formulates occlusion modeling as a rankable retrieval task and introduces a fruit-centric leaf-set attention architecture. Leveraging instance-segmented organ point clouds, the method employs a direction-aware graph neural network combined with joint hierarchical aggregation to construct a scene graph encoding both physical connectivity and directional occlusion relationships. Evaluated on a synthetic pepper dataset, the approach achieves an occlusion prediction F1 score of 0.73 and NDCG@3 of 0.85, with attachment relation inference reaching an edge F1 of 0.83—significantly outperforming ablated variants.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Scene Analysis & Understanding
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Robotic harvesting in dense crop canopies requires effective interventions that depend not only on geometry, but also on explicit, direction-conditioned relations identifying which organs obstruct a target fruit. We present SG-DOR (Scene Graphs with Direction-Conditioned Occlusion Reasoning), a relational framework that, given instance-segmented organ point clouds, infers a scene graph encoding physical attachments and direction-conditioned occlusion. We introduce an occlusion ranking task for retrieving and ranking candidate leaves for a target fruit and approach direction, and propose a direction-aware graph neural architecture with per-fruit leaf-set attention and union-level aggregation. Experiments on a multi-plant synthetic pepper dataset show improved occlusion prediction (F1=0.73, NDCG@3=0.85) and attachment inference (edge F1=0.83) over strong ablations, yielding a structured relational signal for downstream intervention planning.

Problem

• robotic harvesting
• occlusion reasoning
• direction-conditioned relations
• scene graph
• pepper plants

Innovation

• direction-conditioned occlusion
• scene graph
• graph neural network
• occlusion ranking
• robotic harvesting