ODE-Based Transformer Decoders for Iterative Sign Language Translation

2026-08-11

AI Summary

This work addresses the high computational cost and low parameter efficiency of current sign language translation models, which often rely on scaling up model size for performance gains. The authors propose a novel reconstruction of the Transformer decoder from the perspective of ordinary differential equations (ODEs), introducing higher-order numerical integration schemes—such as Runge-Kutta methods RK-2 and RK-4—into sign language translation for the first time. This approach replaces conventional residual connections with more accurate and stable iterative optimization, enhancing model expressiveness without increasing parameter count. Experimental results demonstrate that the proposed method achieves BLEU-4 scores of 22.96 and 19.34 on the PHOENIX-2014-T and CSL-Daily datasets, respectively, outperforming baseline models while using fewer decoder layers and iteration steps.

Abstract

Sign language translation has achieved strong results with Transformer architectures, yet recent improvements largely rely on scaling model capacity at the cost of increased computation. We propose a parameter-efficient alternative that improves expressiveness without increasing model size. Rather than scaling capacity, we focus on enhancing the update dynamics of iterative refinement decoders, where each refinement step corresponds to one internal decoder iteration that progressively improves the latent representation before translation generation. We reinterpret residual refinement updates from an Ordinary Differential Equation (ODE) perspective and replace them with higher-order numerical integration schemes, namely Runge--Kutta methods (RK-2 and RK-4). These methods perform multiple function evaluations within each refinement step to produce more accurate and stable representation updates without adding decoder parameters. To the best of our knowledge, this is the first application of ODE-inspired update dynamics to sign language translation. RK-2 achieves 22.96 BLEU-4 on the PHOENIX-2014-T test set and 19.34 BLEU-4 on the CSL-Daily test set, outperforming the IPSLT baseline on both benchmarks, with fewer decoder layers and refinement iterations on CSL-Daily. These results suggest that stronger refinement dynamics can improve translation performance under parameter-efficient decoder designs, providing a complementary alternative to conventional model scaling.

Problem

• Sign Language Translation
• Parameter Efficiency
• Iterative Refinement
• Transformer Decoders
• Model Scaling

Innovation

• ODE-based update
• Runge-Kutta methods
• iterative refinement
• parameter-efficient Transformer
• sign language translation