A Low-Cost Hybrid Reservoir Computing Model for Isolated Sign Language Video Recognition

2026-08-04

AI Summary

This work addresses the high computational cost and limited deployability of existing deep learning approaches for isolated sign language recognition on edge devices by proposing a lightweight hybrid reservoir computing model. The method leverages MediaPipe to extract hand and body keypoints, then efficiently captures spatiotemporal dynamics by integrating Deep Reservoir Computing (DRC) with Bidirectional Reservoir Computing (BRC), followed by fast classification via ridge regression. Evaluated on the WLASL100 dataset, the model achieves 61.12% Top-1, 86.05% Top-5, and 92.56% Top-10 accuracy with only a few seconds of training time. This approach substantially reduces computational resource requirements while maintaining competitive recognition performance, making it well-suited for deployment on resource-constrained edge devices.

Abstract

Sign language recognition (SLR) enhances communication between hearing and hearing-impaired individuals. Although deep learning (DL) has achieved promising performance in SLR, its high computational cost limits deployment on edge devices. To address this challenge, we propose a lightweight reservoir computing (RC)-based approach for SLR. In the proposed method, MediaPipe extracts body and hand keypoints to capture the spatial and temporal dynamics of gestures. These keypoints are then processed by a hybrid reservoir computing (HRC) architecture that combines deep reservoir computing (DRC) and bidirectional reservoir computing (BRC), transforming the input into a high-dimensional dynamic representation. A ridge regression model maps the final HRC state to class labels. This HRC-based SLR method achieved Top-1, Top-5, and Top-10 accuracies of 61.12%, 86.05%, and 92.56%, respectively, on the Word-Level American Sign Language 100 (WLASL100) video dataset, demonstrating competitive performance compared to deep learning-based approaches. Additionally, due to the lightweight nature of RC, the training time was drastically reduced to only a few seconds compared with DL-based methods such as Bi-GRU.This method offers low computational cost, showing its potential for deployment on edge devices.

Problem

• sign language recognition
• computational cost
• edge devices
• video recognition
• isolated sign language

Innovation

• Reservoir Computing
• Sign Language Recognition
• Hybrid Architecture
• Edge Deployment
• Keypoint-based Representation