Context-Aware Autoregressive Diffusion for Gloss-Wise Sign Language Production

2026-06-19

AI Summary

This work addresses the challenges of temporal drift and gestural ambiguity in existing sign language generation methods, particularly when synthesizing long sentences, which often hinder precise morpheme-level control. To overcome these limitations, the authors propose GARD, a morpheme-level autoregressive diffusion model that introduces diffusion mechanisms into morpheme-level sign synthesis for the first time. GARD jointly models semantic and motion contexts to capture co-articulation effects and incorporates an Inter-Gloss Transition Guidance module alongside a Global Motion Harmonizer to simultaneously optimize boundary alignment and motion fluency. Extensive experiments on the Phoenix-14T and CSL-Daily datasets demonstrate that GARD significantly outperforms current state-of-the-art approaches in both semantic accuracy and motion similarity.

Abstract

To generate natural and accurate sentence-level sign language, synthesizing the "gloss", the fundamental semantic unit, is essential. However, most current sign-language production (SLP) methods generate entire sequences at once. While this end-to-end approach is often efficient, it is prone to temporal drift and hand motion blur as sentences get longer, and fails to accurately control individual glosses. In this paper, we propose the Context-aware Gloss-wise AutoRegressive Diffusion model (GARD), a gloss-wise diffusion framework that models coarticulation by conditioning on both semantic (linguistic) and kinematic (motion) contexts. To ensure natural continuity between gloss motions, GARD introduces two additional strategies: i) Inter-Gloss Transition Guidance, which applies gradient-based guidance to kinematically align inter-gloss boundaries and ensure seamless pose consistency. ii) Global Motion Harmonizer, refining the entire gloss motion sequence based on the boundary poses adjusted by Inter-Gloss Transition Guidance. Extensive experiments on Phoenix-T and CSL-Daily datasets demonstrate that GARD achieves superior performance over existing SLP methods in terms of both linguistic accuracy and motion similarity.

Problem

• sign language production
• gloss-wise generation
• temporal drift
• motion blur
• coarticulation

Innovation

• gloss-wise generation
• autoregressive diffusion
• coarticulation modeling
• inter-gloss transition guidance
• sign language production