Leveraging Computation of Expectation Models for Commonsense Affordance Estimation on 3D Scene Graphs

2024-09-09

IEEE/RJS International Conference on Intelligent RObots and Systems

AI Summary

This work addresses the challenge of affordance recognition for embodied robot task planning in urban environments, where sparse 3D scenes hinder commonsense understanding of object functionality. We propose the Correlation-Expected Conceptual Inference (CECI) model—the first to integrate expectation probability modeling with graph convolutional networks (GCNs) for fine-grained, intra-class semantic affordance estimation grounded in 3D scene graphs. Our method jointly leverages probabilistic distribution learning and structured scene reasoning, eliminating the need for dense supervision. Evaluated on real indoor environments, CECI achieves strong alignment with human commonsense judgments (Cohen’s κ = 0.87) and outperforms existing baselines by +12.6% in mean Average Precision (mAP). The core contribution is the first task-driven, generalizable probabilistic affordance reasoning framework, significantly enhancing robots’ understanding of intrinsic object functionality and enabling more effective task optimization.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

This article studies the commonsense object affordance concept for enabling close-to-human task planning and task optimization of embodied robotic agents in urban environments. The focus of the object affordance is on reasoning how to effectively identify object’s inherent utility during the task execution, which in this work is enabled through the analysis of contextual relations of sparse information of 3D scene graphs. The proposed framework develops a Correlation Information (CECI) model to learn probability distributions using a Graph Convolutional Network, allowing to extract the commonsense affordance for individual members of a semantic class. The overall framework was experimentally validated in a real-world indoor environment, showcasing the ability of the method to level with human commonsense. For a video of the article, showcasing the experimental demonstration, please refer to the following link: https://youtu.be/BDCMVx2GiQE

Problem

• Estimate commonsense affordance for robotic task planning
• Reason object utility via 3D scene graph relations
• Learn affordance distributions using Graph Convolutional Network

Innovation

• Uses Graph Convolutional Network for affordance learning
• Analyzes contextual relations in 3D scene graphs
• Develops CECI model for probability distribution learning