UOL@IDEM at BEA 2026 Shared Task 1: Neural Fusion and Feature-Rich Modeling for L1-Aware Vocabulary Difficulty Prediction

2026-06-23

AI Summary

This study addresses the challenge of predicting second-language (L2) lexical difficulty for learners with distinct native language (L1) backgrounds—specifically Spanish, German, and Chinese speakers. Framing the task as a regression problem, the work proposes an innovative model that integrates multilingual sentence encoders (BGE-M3, mE5, LaBSE) with linguistically informed handcrafted features, including word frequency, surface form properties, cognate similarity, semantic alignment, and masked language model (MLM) predictability. This hybrid approach effectively captures L1-specific influences on L2 vocabulary acquisition difficulty. Evaluated on official test sets, the model significantly outperforms baseline systems, achieving RMSE scores of 1.132 (Spanish), 1.037 (German), and 0.891 (Chinese), demonstrating strong cross-linguistic generalization, with only slight overestimation observed for extremely easy words.

Abstract

This paper describes UOL@IDEM's closed-track submission to the BEA 2026 shared task on L1-aware vocabulary difficulty prediction. We model the task as regression and train separate systems for Spanish, German, and Mandarin Chinese\footnote{Below we use \emph{Chinese} for brevity.}. Our system combines multilingual contextual representations with engineered features capturing frequency, surface form, retrieval evidence, semantic alignment, cognate similarity, and masked-language-model predictability. Development results show consistent gains over the official closed-track baselines, with sentence-embedding encoders such as BGE-M3, multilingual E5, and LaBSE performing best. Official submissions achieve RMSE scores of 1.132, 1.037, and 0.891 for Spanish, German, and Chinese, respectively. Feature analysis identifies frequency as the most stable predictor, while contextual predictability, form similarity, retrieval, and semantic features provide complementary L1-sensitive signals. Error analysis shows strong ranking performance but weaker calibration for the easiest items, which are often overpredicted. See https://github.com/Nouran-Khallaf/UoL-IDEM-BEA2026-Vocabulary-Difficulty-Prediction

Problem

• vocabulary difficulty prediction
• L1-aware
• language learning
• lexical complexity
• multilingual NLP

Innovation

• L1-aware vocabulary difficulty
• neural fusion
• feature-rich modeling
• multilingual contextual representations
• regression-based prediction