Thunder-KoNUBench: A Corpus-Aligned Benchmark for Korean Negation Understanding

2026-01-08

arXiv.org

AI Summary

This study addresses the poor performance of current large language models (LLMs) on Korean negation understanding, primarily attributed to the absence of evaluation benchmarks that reflect real-world linguistic distributions. To bridge this gap, the authors construct Thunder-KoNUBench, the first sentence-level benchmark for Korean negation comprehension, grounded in large-scale corpus analysis. They systematically evaluate 47 LLMs on this benchmark, revealing the nuanced effects of model scale and instruction tuning on negation understanding. Furthermore, they demonstrate that targeted fine-tuning on Thunder-KoNUBench not only substantially improves models’ ability to handle Korean negation but also enhances their general contextual comprehension. This benchmark thus provides a critical resource and a new direction for advancing Korean language understanding research.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Knowledge Representation Languages

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Although negation is known to challenge large language models (LLMs), benchmarks for evaluating negation understanding, especially in Korean, are scarce. We conduct a corpus-based analysis of Korean negation and show that LLM performance degrades under negation. We then introduce Thunder-KoNUBench, a sentence-level benchmark that reflects the empirical distribution of Korean negation phenomena. Evaluating 47 LLMs, we analyze the effects of model size and instruction tuning, and show that fine-tuning on Thunder-KoNUBench improves negation understanding and broader contextual comprehension in Korean.

Problem

• negation understanding
• Korean language
• large language models
• benchmark
• corpus-aligned

Innovation

• Korean negation
• corpus-aligned benchmark
• large language models
• fine-tuning
• negation understanding