ltzGLUE: Luxembourgish General Language Understanding Evaluation

2026-04-20

AI Summary

This work addresses the longstanding absence of a standardized natural language understanding (NLU) evaluation benchmark for Luxembourgish, an official language with limited computational resources. We propose ltzGLUE, the first multitask evaluation framework for Luxembourgish, which adapts the GLUE paradigm by integrating both newly constructed and existing datasets to cover core NLU tasks such as named entity recognition, topic classification, and intent classification. Through systematic evaluation of various Luxembourgish pretrained encoder models, this study establishes current performance baselines and fills a critical gap in the availability of a unified NLU benchmark for this low-resource official language. The resulting framework provides a standardized platform and reference point to support and advance future research in Luxembourgish language processing.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Knowledge Representation Languages

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

This paper presents ltzGLUE, the first Natural Language Understanding (NLU) benchmark for Luxembourgish (LTZ) based on the popular GLUE benchmark for English. Although NLU tasks are available for many European languages nowadays, LTZ is one of the official national languages that is often overlooked. We construct new tasks and reuse existing ones to introduce the first official NLU benchmark and accompanying evaluation of encoder models for the language. Our tasks include common natural language processing tasks in binary and multi-class classification settings, including named entity recognition, topic classification, and intent classification. We evaluate various pre-trained language models for LTZ to present an overview of the current capabilities of these models on the LTZ language.

Problem

• Luxembourgish
• Natural Language Understanding
• NLU benchmark
• low-resource language
• language evaluation

Innovation

• ltzGLUE
• Luxembourgish
• Natural Language Understanding
• language benchmark
• pre-trained language models