Interactive Learning of Physical Object Properties Through Robot Manipulation and Database of Object Measurements

2024-04-10

IEEE/RJS International Conference on Intelligent RObots and Systems

AI Summary

This work addresses the challenge of autonomous robotic perception of object physical attributes—namely material, mass, volume, and stiffness. We propose a closed-loop learning framework integrating active manipulation with large-scale object retrieval. Methodologically, we model attribute uncertainty via a Bayesian network and design an action selection policy based on expected information gain, enabling synergistic exploratory manipulation and online database querying. To our knowledge, this is the first approach robust to “deceptive” objects—those exhibiting inconsistent surface appearance and internal physical properties—and featuring autonomous termination of the learning process. Evaluated on 63 real-world objects over 24,000+ interactions, our method significantly outperforms baselines in both attribute inference accuracy and data efficiency. It achieves fully automated, uncertainty-aware digitization of physical attributes.

Technology Category

• Intelligent Robots: Manipulation
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

This work presents a framework for automatically extracting physical object properties, such as material composition, mass, volume, and stiffness, through robot manipulation and a database of object measurements. The framework involves exploratory action selection to maximize learning about objects on a table. A Bayesian network models conditional dependencies between object properties, incorporating prior probability distributions and uncertainty associated with measurement actions. The algorithm selects optimal exploratory actions based on expected information gain and updates object properties through Bayesian inference. Experimental evaluation demonstrates effective action selection compared to a baseline and correct termination of the experiments if there is nothing more to be learned. The algorithm proved to behave intelligently when presented with trick objects with material properties in conflict with their appearance. The robot pipeline integrates with a logging module and an online database of objects, containing over 24,000 measurements of 63 objects with different grippers. All code and data are publicly available, facilitating automatic digitization of objects and their physical properties through exploratory manipulations.

Problem

• Robot Learning
• Physical Properties
• Object Manipulation

Innovation

• Bayesian Networks
• Autonomous Learning
• Object Characterization