🤖 AI Summary
This study addresses the lack of large-scale, language-agnostic, fine-grained visual datasets for sign language recognition that support signer-aware evaluation. To bridge this gap, the authors construct a balanced dataset based on the Hamburg Notation System (HamNoSys 4), comprising 144,000 RGB images contributed by 15 signers across 160 handshape classes. They introduce, for the first time, a dual evaluation protocol incorporating both signer-dependent and leave-one-signer-out (LOSO) settings, and establish reproducible benchmarks using diverse models—including ResNet-18, ViT-B/16, graph convolutional networks, and XGBoost. Experiments on the ASL Fingerspelling Dataset A achieve Top-1 accuracy of 82.20%–87.40% under the LOSO protocol, while also revealing a significant performance drop in cross-signer generalization, highlighting a critical challenge in real-world deployment.
📝 Abstract
Purpose: Fine-grained handshape recognition supports computational sign-language transcription, recognition, and translation, but broad, phonetically defined visual inventories with signer-aware evaluation remain limited. This work introduces a benchmark grounded in the language-independent Hamburg Notation System (HamNoSys). Methods: A balanced dataset of 144,000 RGB images was collected from 15 participants for 160 handshape classes defined by the official HamNoSys 4 Handshapes Chart. ResNet-18 and ViT-B/16 were evaluated as appearance-based models, while a graph convolutional network and XGBoost were evaluated from hand landmarks. Both a class-stratified subject-dependent split and a 15-fold leave-one-subject-out (LOSO) protocol were used. The same model families were additionally assessed on LSWH100 and ASL Fingerspelling Dataset A for external context. Results: The subject-dependent benchmarks established reproducible reference performance across all four model families, whereas LOSO evaluation exposed a substantial reduction when recognition was required to generalise to unseen participants. On ASL Fingerspelling Dataset A, mean LOSO top-1 accuracy ranged from 82.20% to 87.40%. Conclusion: The documented acquisition, curation, and complementary evaluation protocols pro-vide a reproducible resource for fine-grained isolated-handshape research and for developing more accessible sign-language technologies.