MMS Player: an open source software for parametric data-driven animation of Sign Language avatars

2025-07-22

AI Summary

Traditional sign language gloss annotation lacks parallelism, temporal precision, and prosodic information, resulting in stiff and unnatural animations. To address this, we propose Multimodal Symbolic Streams (MMS), a novel representation that explicitly models co-articulated gestures, fine-grained temporal alignment, and prosodic features. Building upon MMS, we present the first open-source, parameterized sign language animation generation system. Implemented with Blender as the rendering engine, it synthesizes natural, high-fidelity virtual human sign animations directly from MMS input. The system offers both command-line and HTTP API interfaces and supports export to video and mainstream 3D formats. Evaluation shows significant improvements in grammatical accuracy and prosodic naturalness of generated signing. Released under the GPL-3.0 license, this work establishes a scalable, reproducible foundation for sign language synthesis research and accessibility applications.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Language and Vision

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

This paper describes the MMS-Player, an open source software able to synthesise sign language animations from a novel sign language representation format called MMS (MultiModal Signstream). The MMS enhances gloss-based representations by adding information on parallel execution of signs, timing, and inflections. The implementation consists of Python scripts for the popular Blender 3D authoring tool and can be invoked via command line or HTTP API. Animations can be rendered as videos or exported in other popular 3D animation exchange formats. The software is freely available under GPL-3.0 license at https://github.com/DFKI-SignLanguage/MMS-Player.

Problem

• Develops open-source software for sign language avatar animation
• Enhances gloss-based representations with timing and inflection data
• Provides Blender integration for animation rendering and export

Innovation

• Open source parametric animation software
• MultiModal Signstream (MMS) representation format
• Blender integration with Python scripts