Co-creation for Sign Language Processing and Machine Translation

2025-03-03

AI Summary

Sign Language Machine Translation (SLMT) research suffers from systemic exclusion of Deaf signers and insufficient user participation, undermining linguistic equity and sign language sovereignty. Method: We conduct a systematic literature review (110 papers), participatory design workshops, interdisciplinary collaborative analysis, and domain ontology modeling to diagnose gaps in data collection, task formulation, and evaluation criteria. Contribution/Results: We propose the first dynamic, lifecycle-oriented co-creation typology specifically for SLMT and develop NLP’s first formally endorsed Sign Language Co-Creation Guideline. Our framework identifies critical user-exclusion patterns across the SLMT pipeline and provides actionable, stage-specific co-creation pathways—enhancing technical fairness, methodological rigor, and community agency. This work establishes foundational infrastructure for ethically grounded, linguistically appropriate SLMT development.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Life-Long and Continual Learning
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Sign language machine translation (SLMT) -- the task of automatically translating between sign and spoken languages or between sign languages -- is a complex task within the field of NLP. Its multi-modal and non-linear nature require the joint efforts of sign language (SL) linguists, technical experts and SL users. Effective user involvement is a challenge that can be addressed through co-creation. Co-creation has been formally defined in many fields, e.g. business, marketing, educational and others, however in NLP and in particular in SLMT there is no formal, widely accepted definition. Starting from the inception and evolution of co-creation across various fields over time, we develop a relationship typology to address the collaboration between deaf, Hard of Hearing and hearing researchers and the co-creation with SL-users. We compare this new typology to the guiding principles of participatory design for NLP. We, then, assess 110 articles from the perspective of involvement of SL users and highlight the lack of involvement of the sign language community or users in decision-making processes required for effective co-creation. Finally, we derive formal guidelines for co-creation for SLMT which take the dynamic nature of co-creation throughout the life cycle of a research project into account.

Problem

• Defining co-creation in sign language machine translation (SLMT).
• Addressing the lack of SL user involvement in SLMT research.
• Developing guidelines for effective co-creation in SLMT projects.

Innovation

• Developed co-creation typology for SLMT collaboration
• Assessed 110 articles for SL user involvement
• Derived formal co-creation guidelines for SLMT