UKTA: Unified Korean Text Analyzer

2025-02-11

AI Summary

Existing Korean automated writing evaluation (AWE) tools suffer from inadequate multi-perspective analysis, poor error propagation mitigation, and limited interpretability, resulting in inefficient assessment and delayed feedback. To address these limitations, this paper proposes the first multi-level AWE system specifically designed for Korean writing. Our approach innovatively integrates low-level morphological analysis, mid-level interpretable lexical feature modeling, and high-level educationally grounded rubric-driven scoring. It employs rule-enhanced morphological segmentation, domain-adapted token representation, and hierarchical joint regression/classification modeling. Experimental results demonstrate that the system significantly outperforms established baselines across multiple Korean AWE tasks, achieving substantial improvements in accuracy and quadratic weighted kappa. Moreover, it exhibits high inter-rater consistency and robustness suitable for industrial-scale deployment.

Technology Category

• Natural Language Processing: Lexical Semantics and Morphology
• Machine Learning: Evaluation and Analysis
• Knowledge Representation and Reasoning: Knowledge Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Evaluating writing quality is complex and time-consuming often delaying feedback to learners. While automated writing evaluation tools are effective for English, Korean automated writing evaluation tools face challenges due to their inability to address multi-view analysis, error propagation, and evaluation explainability. To overcome these challenges, we introduce UKTA (Unified Korean Text Analyzer), a comprehensive Korea text analysis and writing evaluation system. UKTA provides accurate low-level morpheme analysis, key lexical features for mid-level explainability, and transparent high-level rubric-based writing scores. Our approach enhances accuracy and quadratic weighted kappa over existing baseline, positioning UKTA as a leading multi-perspective tool for Korean text analysis and writing evaluation.

Problem

• Automated evaluation of Korean writing quality
• Addressing multi-view analysis and error propagation
• Enhancing evaluation explainability and accuracy

Innovation

• Unified Korean Text Analyzer
• Multi-view analysis enhancement
• Rubric-based transparent scoring