Continuous Wrist Control on the Hannes Prosthesis: a Vision-based Shared Autonomy Framework

2025-02-24

AI Summary

Most existing prosthetic hands emphasize dexterous finger control while neglecting wrist degrees of freedom, forcing users to compensate via shoulder or elbow movements—resulting in unnatural motion. This work proposes a continuous vision-guided shared-autonomy control framework for the Hannes biomimetic hand, specifically targeting wrist articulation. It is the first to tightly integrate real-time monocular RGB target tracking, user intent decoding, PID-based closed-loop wrist control, and low-level hardware interfacing into a unified architecture, enabling seamless collaboration between autonomous and user-intent-driven wrist pose regulation. The framework supports natural “approach–grasp” motion sequencing and eliminates compensatory movements induced by fixed-wrist configurations. Deployed and validated on a physical Hannes prosthesis, the system achieves end-to-end latency under 80 ms across all modules, significantly improving movement naturalness during grasp preparation and enhancing task completion efficiency.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Biometrics, Face, Gesture & Pose
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Most control techniques for prosthetic grasping focus on dexterous fingers control, but overlook the wrist motion. This forces the user to perform compensatory movements with the elbow, shoulder and hip to adapt the wrist for grasping. We propose a computer vision-based system that leverages the collaboration between the user and an automatic system in a shared autonomy framework, to perform continuous control of the wrist degrees of freedom in a prosthetic arm, promoting a more natural approach-to-grasp motion. Our pipeline allows to seamlessly control the prosthetic wrist to follow the target object and finally orient it for grasping according to the user intent. We assess the effectiveness of each system component through quantitative analysis and finally deploy our method on the Hannes prosthetic arm. Code and videos: https://hsp-iit.github.io/hannes-wrist-control.

Problem

• Continuous wrist control in prosthetics
• Vision-based shared autonomy framework
• Natural approach-to-grasp motion enhancement

Innovation

• Vision-based shared autonomy
• Continuous wrist control
• Natural approach-to-grasp motion