YallaMorph: A Benchmark for Evaluating Arabic Morphological Generation in Large Language Models

2026-09-09

AI Summary

本文针对阿拉伯语形态生成问题，提出YallaMorph基准测试集，用于评估大型语言模型在显式词法和特征输入下的控制生成能力。

Abstract

Arabic morphology remains challenging for large language models, since fluent generation does not guarantee accurate morphosyntactic control. Existing Arabic evaluations mainly target downstream tasks and do not directly test controlled morphological generation from explicit lexical and feature-based input. We introduce YallaMorph, a large-scale benchmark for Arabic morphological generation covering verbs, nouns, adjectives, their cliticized forms, and invalid configurations. We evaluate multilingual and Arabic-oriented LLMs under diacritized and undiacritized settings over 600K benchmark entries. Results show that Arabic morphological generation remains difficult, especially for cliticized, unseen, and morphologically rare forms.

Problem

• Arabic morphology
• large language models
• morphosyntactic control
• controlled morphological generation
• benchmark

Innovation

• YallaMorph
• Arabic Morphological Generation
• Large Language Models
• Benchmark