Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian

2026-06-30

AI Summary

This study addresses the challenge of relation extraction in low-resource languages such as Romanian, where performance is hindered by scarce annotated data. It presents the first systematic evaluation of large language models’ (LLMs) cross-lingual transfer capabilities for this task, leveraging automatic LLM-based translation of English benchmark datasets. The authors evaluate Gemma 2 31B under zero-shot, few-shot, and QLoRA fine-tuning settings, comparing it against encoder-based models including XLM-RoBERTa and Romanian BERT. Experimental results demonstrate that QLoRA fine-tuning improves macro F1 by over 22 percentage points, reducing the performance gap between English and Romanian to just 1.4 percentage points. Notably, a smaller monolingual Romanian BERT matches or exceeds larger multilingual models, suggesting limited advantages of extremely large models in single-task scenarios. The translated dataset and trained models are publicly released.

Abstract

Relation extraction (RE) for low-resource languages is typically constrained by the lack of annotated corpora. We investigate the feasibility of cross-lingual RE for Romanian by combining automatic dataset translation with large language model (LLM) inference. We translate the SemEval-2010 Task 8 benchmark from English to Romanian using an LLM-based translation pipeline and evaluate Gemma 4 31B under zero-shot, few-shot, and QLoRA fine-tuned configurations, against four encoder baselines spanning 125M to 560M parameters: XLM- RoBERTa (base and large), Romanian BERT, and RoBERT- large. We assess two task formulations: relation classification with marked entities and end-to-end extraction. Our results show that Romanian incurs a 3 to 5 percentage point (pp) drop relative to English in prompt-only settings, that few-shot prompting provides marginal gains over zero-shot, and that QLoRA fine-tuning improves macro F1-Score by more than 22 percentage points in both languages while reducing the cross-lingual gap from 3.3 to 1.4pp. The encoder baselines come within 1-4pp of QLoRA Gemma on Romanian despite being 50-250 times smaller, with monolingual Romanian BERT at 125M parameters matching multilingual XLM-R at 278M. The case for using a 31B model for single-task RE on Romanian is therefore weak in deployment scenarios where compute matters. We release the translated dataset, evaluation code, and trained models.

Problem

• Relation Extraction
• Low-resource Languages
• Cross-lingual Transfer
• Romanian
• Annotated Corpora

Innovation

• cross-lingual relation extraction
• large language models
• QLoRA fine-tuning
• low-resource languages
• automatic dataset translation