Translation as a Computationally Efficient Bridge: Feasibility of English BERT for Low-Resource Languages

2026-07-14

AI Summary

This study addresses the challenge of training high-quality BERT models for low-resource languages, which suffer from scarce annotated data and high computational costs. The authors propose an efficient alternative: machine-translating low-resource language data into English and directly fine-tuning a pre-trained English BERT model. They systematically evaluate this approach across six natural language understanding tasks—including sentiment analysis, question answering, and named entity recognition—using data from Bulgarian, Chinese, Dutch, Italian, and Russian. Results show that in 53.3% of the experimental settings, the proposed method matches or surpasses native-language baselines, with particularly strong performance on question answering, part-of-speech tagging, and natural language inference. The findings highlight the critical influence of task type and linguistic proximity on cross-lingual transfer effectiveness, offering a practical and effective solution for low-resource scenarios.

Abstract

BERT models have revolutionised Natural Language Processing (NLP) through their ability to process unstructured text across diverse domains. However, developing high-quality BERT models for non-English languages remains challenging due to limited annotated data and high computational demands. Translating non-English data into English and fine-tuning existing English BERT models offers a resource-efficient alternative, yet few studies have structurally compared translation-based fine-tuning with native-language BERT performance across tasks and languages. This study provides such a comparison, evaluating the feasibility of translation-based fine-tuning across six NLP tasks: Sentiment Analysis, Hate Speech Detection, Question Answering, Named Entity Recognition, Part-of-Speech Tagging, and Natural Language Inference, using datasets translated from Bulgarian, Chinese, Dutch, Italian, and Russian. Across all settings, the translation-based approach was comparable or superior in 53.3 percent of cases. Gains were most frequent in Question Answering, Part-of-Speech Tagging, and Natural Language Inference, while performance declines were common in Named Entity Recognition and Hate Speech Detection. The results show that translation-based fine-tuning is most effective for tasks relying on syntactic or structural patterns and for languages typologically close to English, such as Dutch, but less effective for token-level or culturally nuanced tasks, particularly in Chinese. Overall, this study demonstrates that translation-based fine-tuning offers a scalable, resource-efficient, and empirically validated path for extending NLP to low-resource languages while advancing linguistic inclusivity and sustainability in artificial intelligence.

Problem

• low-resource languages
• BERT
• machine translation
• cross-lingual transfer
• NLP

Innovation

• translation-based fine-tuning
• low-resource languages
• English BERT
• cross-lingual transfer
• computational efficiency