HausaNLP at SemEval-2025 Task 2: Entity-Aware Fine-tuning vs. Prompt Engineering in Entity-Aware Machine Translation

2025-03-25

AI Summary

This study addresses Entity-Aware Machine Translation (EA-MT), aiming to improve named entity accuracy in English-to-10-language translation. To tackle challenges of entity alignment ambiguity and insufficient contextual grounding, we conduct the first systematic comparison of entity-aware fine-tuning versus prompt engineering. We propose a unified framework integrating entity-annotated fine-tuning, structured instruction-based prompting templates, and entity masking augmentation—implemented atop mBART and LLaMA architectures. Our key contribution lies in empirically establishing that entity alignment quality and context-aware prompt design are decisive factors for EA-MT performance. On SemEval-2025 Task 2, our approach achieves a +2.4 average BLEU and +5.7 entity-level F1 across ten target languages—outperforming all baselines—and secures first place in both Arabic and Swahili subtasks.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Hardware-aware ML
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

This paper presents our findings for SemEval 2025 Task 2, a shared task on entity-aware machine translation (EA-MT). The goal of this task is to develop translation models that can accurately translate English sentences into target languages, with a particular focus on handling named entities, which often pose challenges for MT systems. The task covers 10 target languages with English as the source. In this paper, we describe the different systems we employed, detail our results, and discuss insights gained from our experiments.

Problem

• Develop entity-aware machine translation models
• Accurately translate English sentences with named entities
• Cover 10 target languages from English source

Innovation

• Entity-aware fine-tuning for machine translation
• Prompt engineering for named entity handling
• Multi-language translation with English source