Hierarchical Procedural Framework for Low-latency Robot-Assisted Hand-Object Interaction

2024-05-29

AI Summary

To address inaccurate human motion perception and delayed adaptive control in human–robot interaction (HRI), leading to poor coordination, this paper proposes a hierarchical procedural assistive control framework tailored for hand–object interaction scenarios. Methodologically, it integrates vision-driven real-time 3D hand reconstruction with motion primitive modeling, enabling dual-layer (open-loop feedforward and closed-loop feedback) mapping from human actions to robot responses. A novel open-loop–closed-loop dynamic coordination mechanism is introduced. The framework synergistically combines computer vision, low-latency communication, and hierarchical feedback control. Experimental evaluation demonstrates an end-to-end latency of ≤0.3 seconds; validation in in-the-loop wearing tasks confirms its effectiveness and practicality in medical assistive devices and intelligent manufacturing applications. Results show significant improvements in human–robot motion synchronization accuracy and real-time adaptability.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Interaction Techniques and Devices
• Computer Vision: Low Level & Physics-based Vision

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Advances in robotics have been driving the development of human-robot interaction (HRI) technologies. However, accurately perceiving human actions and achieving adaptive control remains a challenge in facilitating seamless coordination between human and robotic movements. In this paper, we propose a hierarchical procedural framework to enable dynamic robot-assisted hand-object interaction. An open-loop hierarchy leverages the computer vision (CV)-based 3D reconstruction of the human hand, based on which motion primitives have been designed to translate hand motions into robotic actions. The low-level coordination hierarchy fine-tunes the robot's action by using the continuously updated 3D hand models. Experimental validation demonstrates the effectiveness of the hierarchical control architecture. The adaptive coordination between human and robot behavior has achieved a delay of $leq 0.3$ seconds in the tele-interaction scenario. A case study of ring-wearing tasks indicates the potential application of this work in assistive technologies such as healthcare and manufacturing.

Problem

• Enable low-latency robot-assisted hand-object interaction
• Achieve adaptive control in human-robot coordination
• Translate hand motions into robotic actions accurately

Innovation

• Hierarchical framework for low-latency hand-object interaction
• CV-based 3D hand reconstruction for motion translation
• Adaptive coordination with ≤0.3s delay in tele-interaction