Assessing the Impact of Typological Features on Multilingual Machine Translation in the Age of Large Language Models

2026-02-03

AI Summary

This study addresses the uneven translation quality across languages in multilingual machine translation, where the influence of linguistic typological features—beyond data imbalance—remains poorly understood. For the first time, it systematically quantifies the independent impact of target-language typological properties on translation performance using large-scale pretrained models (NLLB-200 and Tower+) and a controlled-variable analysis based on fine-grained typological features from the FLORES+ benchmark covering 212 languages. The findings reveal that specific typological characteristics significantly affect translation quality, with certain languages benefiting notably from an expanded decoding search space. The work also releases a fine-grained typological dataset for all 212 languages and proposes a novel direction for optimizing decoding strategies tailored to linguistic properties.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Evaluation and Analysis

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Despite major advances in multilingual modeling, large quality disparities persist across languages. Besides the obvious impact of uneven training resources, typological properties have also been proposed to determine the intrinsic difficulty of modeling a language. The existing evidence, however, is mostly based on small monolingual language models or bilingual translation models trained from scratch. We expand on this line of work by analyzing two large pre-trained multilingual translation models, NLLB-200 and Tower+, which are state-of-the-art representatives of encoder-decoder and decoder-only machine translation, respectively. Based on a broad set of languages, we find that target language typology drives translation quality of both models, even after controlling for more trivial factors, such as data resourcedness and writing script. Additionally, languages with certain typological properties benefit more from a wider search of the output space, suggesting that such languages could profit from alternative decoding strategies beyond the standard left-to-right beam search. To facilitate further research in this area, we release a set of fine-grained typological properties for 212 languages of the FLORES+ MT evaluation benchmark.

Problem

• typological features
• multilingual machine translation
• translation quality
• large language models
• language disparity

Innovation

• typological features
• multilingual machine translation
• large language models
• decoding strategies
• translation quality