RUMLEM: A Dictionary-Based Lemmatizer for Romansh

2026-04-13

AI Summary

This study addresses the challenges of lemmatization, dialect identification, and language classification for Romansh—a language encompassing five major dialects and the standardized variety Rumantsch Grischun—by proposing the first unified framework covering all its variants. Leveraging a community-built morphological database, the authors develop a variant-aware, lexicon-based lemmatizer that employs rule-based matching to achieve high lemmatization coverage. This approach is further extended to support dialect identification and language boundary detection. Experimental results demonstrate lemmatization coverage of 77–84% on typical texts, dialect identification accuracy of 95% across a corpus of 30,000 documents, and effective discrimination between Romansh and non-Romansh texts.

Technology Category

• Natural Language Processing: Lexical Semantics and Morphology
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Knowledge Representation Languages

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Lemmatization -- the task of mapping an inflected word form to its dictionary form -- is a crucial component of many NLP applications. In this paper, we present RUMLEM, a lemmatizer that covers the five main varieties of Romansh as well as the supra-regional standard variety Rumantsch Grischun. It is based on comprehensive, community-driven morphological databases for Romansh, enabling RUMLEM to cover 77-84% of the words in a typical Romansh text. Since there is a dedicated database for each Romansh variety, an additional application of RUMLEM is variety-aware language classification. Evaluation on 30'000 Romansh texts of varying lengths shows that RUMLEM correctly identifies the variety in 95% of cases. In addition, a proof of concept demonstrates the feasibility of Romansh vs. non-Romansh language classification based on the lemmatizer.

Problem

• lemmatization
• Romansh
• morphological databases
• language variety identification
• NLP

Innovation

• lemmatization
• Romansh
• morphological database
• variety-aware classification
• dictionary-based NLP