A Causal Bayesian Network and Probabilistic Programming Based Reasoning Framework for Robot Manipulation Under Uncertainty

2024-03-21

AI Summary

To address the lack of causal reasoning under uncertainty in robotic manipulation, this paper proposes the first physics-informed inference framework integrating causal Bayesian networks (CBNs) with probabilistic programming (Pyro/Stan). The method embeds physical priors into the causal graph structure, enabling joint uncertainty modeling across perception, state estimation, and execution, while supporting greedy sequential decision-making. Its core contribution is the first synergistic use of CBNs and probabilistic programming for causal reasoning in robotic manipulation—uniquely balancing causal semantics with system generalizability. In Gazebo-simulated block-stacking tasks, the approach achieves 88.6% action prediction accuracy and 94.2% task success rate. Feasibility is further validated on a real-world domestic robot platform.

Technology Category

• Reasoning under Uncertainty: Causality
• Intelligent Robots: Manipulation
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Robot object manipulation in real-world environments is challenging because robot operation must be robust to a range of sensing, estimation, and actuation uncertainties to avoid potentially unsafe and costly mistakes that are a barrier to their adoption. In this paper, we propose a flexible and generalisable physics-informed causal Bayesian network (CBN) based framework for a robot to probabilistically reason about candidate manipulation actions, to enable robot decision-making robust to arbitrary robot system uncertainties -- the first of its kind to use a probabilistic programming language implementation. Using experiments in high-fidelity Gazebo simulation of an exemplar block stacking task, we demonstrate our framework's ability to: (1) predict manipulation outcomes with high accuracy (Pred Acc: 88.6%); and, (2) perform greedy next-best action selection with 94.2% task success rate. We also demonstrate our framework's suitability for real-world robot systems with a domestic robot. Thus, we show that by combining probabilistic causal modelling with physics simulations, we can make robot manipulation more robust to system uncertainties and hence more feasible for real-world applications. Further, our generalised reasoning framework can be used and extended for future robotics and causality research.

Problem

• Robots need causal reasoning for object manipulation tasks
• Data-driven approaches often lack causal semantics, relying on correlations
• Handling real-world uncertainty in robot manipulation is challenging

Innovation

• Causal Bayesian networks for robot manipulation
• Probabilistic programming for interventional inference
• Sim2real transfer handling real-world uncertainty