Translating Signals to Languages for sEMG-Based Activity Recognition

2026-05-21

AI Summary

This work addresses the insufficient semantic modeling of surface electromyography (sEMG) signals in human activity recognition by proposing LLM-sEMG, a novel framework that introduces large language models (LLMs) into sEMG understanding for the first time. The approach employs a language-guided mapping mechanism to transform continuous sEMG sequences into language-like symbolic sequences, thereby leveraging the rich action-related semantic knowledge embedded in pre-trained LLMs for intent recognition. Experimental results demonstrate that LLM-sEMG achieves high-accuracy activity recognition across multiple public sEMG datasets, confirming the effectiveness and generalization capability of large language models in interpreting non-linguistic physiological signals.

Abstract

Surface electromyography (sEMG) signal-based activity recognition has attracted increasing research attention in recent years. To develop accurate sEMG signal-based activity recognizers, numerous approaches have been proposed. Some studies focus on designing larger and more expressive model architectures to enhance the representational capacity of sEMG signals, while others aim to enrich model priors through large-scale pretraining, thereby improving recognition performance. Recently, large language models (LLMs) have shown remarkable generalization and reasoning capabilities in natural language processing, whose implicit knowledge, learned from extensive linguistic descriptions of actions, opens new possibilities for interpreting sEMG signals and inferring activity intentions. Motivated by this, we propose LLM-sEMG, a novel framework that leverages LLMs as sEMG activity recognizers. Within this framework, we design a language-oriented mapping mechanism that converts continuous sEMG sequences into sEMG language, integrating several strategies to further facilitate the signal-to-language mapping process. Extensive experiments demonstrate that the proposed framework achieves highly accurate sEMG signal-based activity recognition using large language models.

Problem

• sEMG
• activity recognition
• large language models
• signal-to-language mapping

Innovation

• large language models
• sEMG signal
• signal-to-language mapping
• activity recognition
• language-oriented representation