GrammaMT: Improving Machine Translation with Grammar-Informed In-Context Learning

2024-10-24

arXiv.org

AI Summary

To address the scarcity of parallel corpora for low-resource and endangered language machine translation (MT), this paper proposes GrammaMT—a training-free, grammar-aware zero-shot prompting method that systematically integrates interlinear glossed text (IGT) into the in-context learning (ICL) framework of large language models (LLMs) for the first time. GrammaMT introduces three novel zero-shot prompting strategies—gloss-shot, chain-gloss, and model-gloss—that leverage morphosyntactic annotations without fine-tuning or additional parameters. Evaluated on SIGMORPHON endangered languages, large-scale IGT corpora, and the cross-domain FLORES benchmark, GrammaMT significantly improves translation performance of open-source instruction-tuned LLMs (e.g., Phi-3, Llama-3), achieving gains exceeding 17 BLEU points over strong baselines. Results demonstrate that grammar-structured prompting substantially enhances both effectiveness and generalizability in low-resource MT, offering a scalable, parameter-efficient alternative to conventional supervised or fine-tuning–based approaches.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

We introduce GrammaMT, a grammatically-aware prompting approach for machine translation that uses Interlinear Glossed Text (IGT), a common form of linguistic description providing morphological and lexical annotations for source sentences. GrammaMT proposes three prompting strategies: gloss-shot, chain-gloss and model-gloss. All are training-free, requiring only a few examples that involve minimal effort to collect, and making them well-suited for low-resource setups. Experiments show that GrammaMT enhances translation performance on open-source instruction-tuned LLMs for various low- to high-resource languages across three benchmarks: (1) the largest IGT corpus, (2) the challenging 2023 SIGMORPHON Shared Task data over endangered languages, and (3) even in an out-of-domain setting with FLORES. Moreover, ablation studies reveal that leveraging gloss resources could substantially boost MT performance (by over 17 BLEU points) if LLMs accurately generate or access input sentence glosses.

Problem

• Improving machine translation using grammar-informed in-context learning
• Enhancing translation for low-resource languages with minimal training data
• Leveraging linguistic glosses to boost performance in diverse language benchmarks

Innovation

• Grammar-aware prompting with Interlinear Glossed Text
• Training-free gloss-shot, chain-gloss, model-gloss strategies
• Boosts MT performance by 17 BLEU points