Enriching the Korean Learner Corpus with Multi-reference Annotations and Rubric-Based Scoring

2025-05-01

AI Summary

To address the scarcity of learner corpora, coarse-grained annotations, and inconsistent evaluation criteria in Korean-as-a-second-language (L2) writing research, this study introduces KoLLA, an enhanced Korean L2 writing corpus. KoLLA features the first multi-reference grammatical error correction (GEC) annotation scheme and fine-grained human scoring—aligned with the National Institute of Korean Language’s standardized rubric—for grammatical accuracy, coherence, and lexical diversity. Annotation employs multi-expert collaboration, rubric-driven scoring, inter-annotator consistency checks, and cross-reference discrepancy analysis to balance linguistic variability and assessment objectivity. Empirical evaluation demonstrates that KoLLA significantly improves the robustness of GEC models and the precision of educational outcome quantification. As the first benchmark resource integrating multi-reference GEC annotations with standardized, rubric-based scoring, KoLLA advances Korean L2 writing research, automated assessment, and pedagogical feedback.

Technology Category

• Natural Language Processing: Lexical Semantics and Morphology
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Despite growing global interest in Korean language education, there remains a significant lack of learner corpora tailored to Korean L2 writing. To address this gap, we enhance the KoLLA Korean learner corpus by adding multiple grammatical error correction (GEC) references, thereby enabling more nuanced and flexible evaluation of GEC systems, and reflects the variability of human language. Additionally, we enrich the corpus with rubric-based scores aligned with guidelines from the Korean National Language Institute, capturing grammatical accuracy, coherence, and lexical diversity. These enhancements make KoLLA a robust and standardized resource for research in Korean L2 education, supporting advancements in language learning, assessment, and automated error correction.

Problem

• Lack of tailored Korean L2 writing learner corpora
• Need nuanced evaluation of grammatical error correction
• Require standardized rubric-based scoring for Korean L2

Innovation

• Adding multiple GEC references for evaluation
• Enriching corpus with rubric-based scoring
• Aligning scores with national language guidelines