ChatEMG: Synthetic Data Generation to Control a Robotic Hand Orthosis for Stroke

2024-06-17

IEEE Robotics and Automation Letters

AI Summary

To address the challenges of scarce electromyography (EMG) data and substantial inter-condition/inter-subject signal variability in stroke patients’ hand orthosis control, this paper proposes a prompt-driven few-shot EMG synthesis and robust decoding framework. We introduce, for the first time, a conditional autoregressive Transformer model for EMG time-series generation, augmented with an EMG-specific prompting mechanism that synthesizes high-fidelity samples from minimal real-data exemplars. Subsequently, we establish a few-shot fine-tuning pipeline and a multi-subject generalization evaluation protocol. Experiments demonstrate significant improvements in motion-intent classification accuracy for unseen subjects and sessions (average +12.7%), and successful end-to-end functional orthosis control is validated within a single clinical session. The framework delivers a deployable paradigm for resource-constrained neural rehabilitation interfaces.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Intelligent Robots: State Estimation
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Intent inferral on a hand orthosis for stroke patients is challenging due to the difficulty of data collection. Additionally, EMG signals exhibit significant variations across different conditions, sessions, and subjects, making it hard for classifiers to generalize. Traditional approaches require a large labeled dataset from the new condition, session, or subject to train intent classifiers; however, this data collection process is burdensome and time-consuming. In this letter, we propose ChatEMG, an autoregressive generative model that can generate synthetic EMG signals conditioned on prompts (i.e., a given sequence of EMG signals). ChatEMG enables us to collect only a small dataset from the new condition, session, or subject and expand it with synthetic samples conditioned on prompts from this new context. ChatEMG leverages a vast repository of previous data via generative training while still remaining context-specific via prompting. Our experiments show that these synthetic samples are classifier-agnostic and can improve intent inferral accuracy for different types of classifiers. We demonstrate that our complete approach can be integrated into a single patient session, including the use of the classifier for functional orthosis-assisted tasks. To the best of our knowledge, this is the first time an intent classifier trained partially on synthetic data has been deployed for functional control of an orthosis by a stroke survivor.

Problem

• Generating synthetic EMG signals for stroke patients' hand orthosis control
• Reducing data collection burden for EMG-based intent classification
• Improving classifier generalization across conditions and subjects

Innovation

• Generates synthetic EMG signals using prompts
• Leverages previous data via generative training
• Improves classifier accuracy with synthetic samples