Soft Vision-Based Tactile-Enabled SixthFinger: Advancing Daily Objects Manipulation for Stroke Survivors

2025-01-12

AI Summary

To address hand grasping impairments in post-stroke patients that hinder manipulation of daily objects, this study proposes a soft, vision-guided, and haptically enhanced “sixth finger” assistive robotic system. Methodologically, we establish a vision-informed tactile perception paradigm, integrating multimodal tactile data to train a lightweight Transformer-based slip detection model (latency < 80 ms), and combine soft actuators with an adaptive closed-loop force control algorithm (grip force error < 0.3 N). Experimentally, the system robustly manipulates over ten categories of everyday items—including water cups, keys, and towels—in real-world settings, significantly improving task completion rate and movement naturalness. This work contributes a deployable, multimodal perception–execution integrated framework for neurorehabilitation assistive devices, advancing the state of soft robotics in clinical assistive applications.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Multi-modal Vision
• Humans and AI: Interaction Techniques and Devices

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

The presence of post-stroke grasping deficiencies highlights the critical need for the development and implementation of advanced compensatory strategies. This paper introduces a novel system to aid chronic stroke survivors through the development of a soft, vision-based, tactile-enabled extra robotic finger. By incorporating vision-based tactile sensing, the system autonomously adjusts grip force in response to slippage detection. This synergy not only ensures mechanical stability but also enriches tactile feedback, mimicking the dynamics of human-object interactions. At the core of our approach is a transformer-based framework trained on a comprehensive tactile dataset encompassing objects with a wide range of morphological properties, including variations in shape, size, weight, texture, and hardness. Furthermore, we validated the system's robustness in real-world applications, where it successfully manipulated various everyday objects. The promising results highlight the potential of this approach to improve the quality of life for stroke survivors.

Problem

• Stroke
• Hand Functional Impairment
• Activities of Daily Living

Innovation

• Smart Haptic Feedback
• Adaptive Grip Technology
• Stroke Rehabilitation Aid