Towards Automated Lexicography: Generating and Evaluating Definitions for Learner's Dictionaries

2026-01-05

arXiv.org

AI Summary

This study addresses the high cost of manually crafting dictionary definitions for language learners by proposing the first method that integrates professional lexicographic standards with large language models (LLMs) to automatically generate concise and accurate non-contextual definitions. The approach leverages iterative LLM-based simplification to refine definitions and introduces a novel LLM-as-a-judge framework for automatic evaluation. This evaluation framework demonstrates strong alignment with human judgments and, under newly established criteria, produces definitions that effectively balance conciseness and accuracy, thereby validating both the generation method and the assessment system. Additionally, the work contributes the first Japanese learner’s dictionary definition dataset, facilitating future research in this area.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We study dictionary definition generation (DDG), i.e., the generation of non-contextualized definitions for given headwords. Dictionary definitions are an essential resource for learning word senses, but manually creating them is costly, which motivates us to automate the process. Specifically, we address learner's dictionary definition generation (LDDG), where definitions should consist of simple words. First, we introduce a reliable evaluation approach for DDG, based on our new evaluation criteria and powered by an LLM-as-a-judge. To provide reference definitions for the evaluation, we also construct a Japanese dataset in collaboration with a professional lexicographer. Validation results demonstrate that our evaluation approach agrees reasonably well with human annotators. Second, we propose an LDDG approach via iterative simplification with an LLM. Experimental results indicate that definitions generated by our approach achieve high scores on our criteria while maintaining lexical simplicity.

Problem

• dictionary definition generation
• learner's dictionary
• lexical simplification
• automated lexicography
• definition generation

Innovation

• dictionary definition generation
• learner's dictionary
• LLM-as-a-judge
• iterative simplification
• lexical simplicity