CART-MPC: Coordinating Assistive Devices for Robot-Assisted Transferring with Multi-Agent Model Predictive Control

2025-01-19

AI Summary

This work addresses the challenge of coordinated multi-device operation in bed-to-wheelchair transfers—encompassing heavy-load manipulation, deformable sling modeling, and complex hook-attachment tasks. We propose a rotation-based multi-agent model predictive control (MPC) framework. Innovatively, we leverage the linking number from knot theory to formulate a differentiable geometric cost function that quantifies sling entanglement states. By integrating keypoint-based neural dynamics models with neural amortization strategies, the framework achieves efficient real-time inference. Evaluated in RCareWorld, our method generalizes across diverse hook configurations, sling materials, and human anatomies. Furthermore, it successfully executes autonomous sling fastening and safe manikin transfer on a physical robotic platform—achieving, for the first time, zero-shot sim-to-real transfer without domain adaptation or fine-tuning.

Technology Category

• Intelligent Robots: Manipulation
• Planning, Routing, and Scheduling: Model-Based Reasoning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Bed-to-wheelchair transferring is a ubiquitous activity of daily living (ADL), but especially challenging for caregiving robots with limited payloads. We develop a novel algorithm that leverages the presence of other assistive devices: a Hoyer sling and a wheelchair for coarse manipulation of heavy loads, alongside a robot arm for fine-grained manipulation of deformable objects (Hoyer sling straps). We instrument the Hoyer sling and wheelchair with actuators and sensors so that they can become intelligent agents in the algorithm. We then focus on one subtask of the transferring ADL -- tying Hoyer sling straps to the sling bar -- that exemplifies the challenges of transfer: multi-agent planning, deformable object manipulation, and generalization to varying hook shapes, sling materials, and care recipient bodies. To address these challenges, we propose CART-MPC, a novel algorithm based on turn-taking multi-agent model predictive control that uses a learned neural dynamics model for a keypoint-based representation of the deformable Hoyer sling strap, and a novel cost function that leverages linking numbers from knot theory and neural amortization to accelerate inference. We validate it in both RCareWorld simulation and real-world environments. In simulation, CART-MPC successfully generalizes across diverse hook designs, sling materials, and care recipient body shapes. In the real world, we show zero-shot sim-to-real generalization capabilities to tie deformable Hoyer sling straps on a sling bar towards transferring a manikin from a hospital bed to a wheelchair. See our website for supplementary materials: https://emprise.cs.cornell.edu/cart-mpc/.

Problem

• Robot-assisted Transfer
• Heavy and Deformable Objects
• Complex Task Coordination

Innovation

• CART-MPC
• Multi-Agent Control
• Deformable Object Handling