A Catalog of Basque Dialectal Resources: Online Collections and Standard-to-Dialectal Adaptations

2026-03-26

AI Summary

This study addresses the severe data scarcity hindering natural language processing (NLP) research on Basque dialects by systematically integrating diverse contemporary dialectal resources for the first time. The work compiles a comprehensive online corpus spanning news, social media, and lexical sources, incorporating both natively generated web content and dialectal texts derived from standard Basque through manual or automatic conversion. A key innovation lies in the creation of a high-quality, manually adapted XNLI dialect test set alongside an automatically generated silver-standard dataset, both covering the three major Basque dialects and validated by native speakers. The resulting parallel evaluation benchmark provides a robust foundation for advancing NLP in this low-resource multilingual setting.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Knowledge Acquisition from the Web

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Recent research on dialectal NLP has identified data scarcity as a primary limitation. To address this limitation, this paper presents a catalog of contemporary Basque dialectal data and resources, offering a systematic and comprehensive compilation of the dialectal data currently available in Basque. Two types of data sources have been distinguished: online data originally written in some dialect, and standard-to-dialect adapted data. The former includes all dialectal data that can be found online, such as news and radio sites, informal tweets, as well as online resources such as dictionaries, atlases, grammar rules, or videos. The latter consists of data that has been adapted from the standard variety to dialectal varieties, either manually or automatically. Regarding the manual adaptation, the test split of the XNLI Natural Language Inference dataset was manually adapted into three Basque dialects: Western, Central, and Navarrese-Lapurdian, yielding a high-quality parallel gold standard evaluation dataset. With respect to the automatic dialectal adaptation, the automatically adapted physical commonsense dataset (BasPhyCowest) underwent additional manual evaluation by native speakers to assess its quality and determine whether it could serve as a viable substitute for full manual adaptation (i.e., silver data creation).

Problem

• dialectal NLP
• data scarcity
• Basque dialects
• dialectal resources
• language adaptation

Innovation

• dialectal NLP
• data scarcity
• manual adaptation
• automatic adaptation
• silver data