Learning Contact Dynamics for Control with Action-conditioned Face Interaction Graph Networks

2025-09-15

AI Summary

To address inaccurate joint prediction of end-effector motion and wrench in contact-intensive robotic manipulation, this paper proposes Action-Conditioned Face Interaction Graph Networks (AFIGNet), an extension of FIGNet that introduces novel node and edge types to explicitly model contact geometry and dynamics coupling. AFIGNet serves as a learnable physics simulator embedded within a Model Predictive Control (MPC) framework, enabling real-time closed-loop control and state estimation. In simulation, it achieves control performance comparable to ground-truth dynamical models; on physical hardware, it reduces motion prediction error by 50% and wrench prediction error by approximately threefold. The core contributions are: (1) the first formulation of face-level contact structure as graph topology, and (2) high-fidelity, action-conditioned learning of contact dynamics—significantly improving prediction accuracy and control robustness in complex, multi-contact scenarios.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Low Level & Physics-based Vision

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We present a learnable physics simulator that provides accurate motion and force-torque prediction of robot end effectors in contact-rich manipulation. The proposed model extends the state-of-the-art GNN-based simulator (FIGNet) with novel node and edge types, enabling action-conditional predictions for control and state estimation tasks. In simulation, the MPC agent using our model matches the performance of the same controller with the ground truth dynamics model in a challenging peg-in-hole task, while in the real-world experiment, our model achieves a 50% improvement in motion prediction accuracy and 3$ imes$ increase in force-torque prediction precision over the baseline physics simulator. Source code and data are publicly available.

Problem

• Predicting robot motion and forces during contact-rich manipulation
• Enabling action-conditional predictions for robot control tasks
• Improving accuracy over baseline physics simulators in real-world applications

Innovation

• Action-conditioned graph networks
• Novel node and edge types
• Learnable physics simulation model