NeuroManip: Prosthetic Hand Manipulation System Based on EMG and Eye Tracking Powered by the Neuromorphic Processor AltAi
🤖 AI Summary
This study addresses the limited contextual awareness of current upper-limb prosthetic control systems, which struggle to balance energy efficiency, safety, and operational precision. To overcome this, the authors propose a context-aware control framework that integrates surface electromyography (sEMG) with eye-tracking–guided computer vision. For the first time, spiking neural networks (SNNs) are combined with gaze-visual contextual cues and deployed on the neuromorphic AltAi processor. Operating at sub-watt power levels, the system enables context-adaptive gesture selection while effectively excluding unsafe grasps. Experimental results demonstrate that the approach achieves gesture recognition accuracy comparable to state-of-the-art sEMG interfaces for six functional gestures; notably, incorporating visual context elevates accuracy to approximately 95%, substantially enhancing both safety and practical usability.
Application Category
📝 Abstract
This paper presents a novel neuromorphic control architecture for upper-limb prostheses that combines surface electromyography (sEMG) with gaze-guided computer vision. The system uses a spiking neural network deployed on the neuromorphic processor AltAi to classify EMG patterns in real time while an eye-tracking headset and scene camera identify the object within the user's focus. In our prototype, the same EMG recognition model that was originally developed for a conventional GPU is deployed as a spiking network on AltAi, achieving comparable accuracy while operating in a sub-watt power regime, which enables a lightweight, wearable implementation. For six distinct functional gestures recorded from upper-limb amputees, the system achieves robust recognition performance comparable to state-of-the-art myoelectric interfaces. When the vision pipeline restricts the decision space to three context-appropriate gestures for the currently viewed object, recognition accuracy increases to roughly 95% while excluding unsafe, object-inappropriate grasps. These results indicate that the proposed neuromorphic, context-aware controller can provide energy-efficient and reliable prosthesis control and has the potential to improve safety and usability in everyday activities for people with upper-limb amputation.
Problem
Research questions and friction points this paper is trying to address.
prosthetic control
upper-limb amputation
context-aware manipulation
energy-efficient
gesture recognition
Innovation
Methods, ideas, or system contributions that make the work stand out.
neuromorphic computing
spiking neural network
EMG-based control
gaze-guided interaction
prosthetic hand
🔎 Similar Papers
No similar papers found.
