ContactSDF: Signed Distance Functions as Multi-Contact Models for Dexterous Manipulation

2024-08-18

IEEE Robotics and Automation Letters

AI Summary

Modeling multi-contact dynamical systems remains challenging due to modeling complexity and poor real-time performance. To address this, we propose an end-to-end differentiable dynamics modeling framework based on Signed Distance Functions (SDFs). Our method introduces a novel dual-SDF architecture: one SDF encodes the supporting plane for efficient collision detection, while the other—coupled with contact dual cones—enables physically consistent, time-stepped state prediction. The entire model is fully differentiable, enabling gradient-based optimization and seamless integration into learning-based control and real-time closed-loop optimization. In simulation, the framework achieves high-fidelity dynamics fitting; on the Allegro dexterous hand, it accomplishes in-hand object reorientation within ≈2 minutes of online learning, operating at 30–60 Hz control frequency. To our knowledge, this is the first fully differentiable, closed-form, and computationally efficient multi-contact dynamics model, establishing a new paradigm for model-based real-time dexterous manipulation control.

Technology Category

• Intelligent Robots: State Estimation
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Machine Learning: Learning with Manifolds

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

In this paper, we propose ContactSDF, a method that uses signed distance functions (SDFs) to approximate multi-contact models, including both collision detection and time-stepping routines. ContactSDF first establishes an SDF using the supporting plane representation of an object for collision detection, and then uses the generated contact dual cones to build a second SDF for time-stepping prediction of the next state. Those two SDFs create a differentiable and closed-form multi-contact dynamic model for state prediction, enabling efficient model learning and optimization for contact-rich manipulation. We perform extensive simulation experiments to show the effectiveness of ContactSDF for model learning and real-time control of dexterous manipulation. We further evaluate the ContactSDF on a hardware Allegro hand for on-palm reorientation tasks. Results show with around 2 minutes of learning on hardware, the ContactSDF achieves high-quality dexterous manipulation at a frequency of 30-60Hz. Project page https://yangwen-1102.github.io/contactsdf.github.io/

Problem

• Develops ContactSDF for multi-contact modeling in manipulation.
• Uses SDFs for collision detection and time-stepping prediction.
• Enables efficient learning and control of dexterous manipulation tasks.

Innovation

• Uses SDFs for multi-contact collision detection
• Creates differentiable dynamic model for state prediction
• Enables efficient learning and real-time manipulation