The CoNLL-2013 Shared Task on Grammatical Error Correction

2025-07-12

AI Summary

This paper addresses grammatical error correction (GEC) for non-native English text. Methodologically, it establishes the first systematic CoNLL-style shared task for GEC, constructing a unified, human-annotated dataset and introducing a standardized evaluation protocol with an open-source scoring toolkit—centered on the F₀.₅ metric to balance precision and recall. The framework integrates rule-based, statistical machine learning, and deep learning approaches, all trained and evaluated on high-quality, manually refined corpora. Fourteen international teams participated, and empirical results demonstrate complementary strengths across methods in precision and error coverage. The work delivers the first reproducible GEC benchmark, accompanied by full public release of data, annotations, evaluation scripts, and baseline systems. This open infrastructure serves as both a technical reference and an accessible platform for future research in GEC.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Evaluation and Analysis
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Large-scale security measurements

Abstract

The CoNLL-2013 shared task was devoted to grammatical error correction. In this paper, we give the task definition, present the data sets, and describe the evaluation metric and scorer used in the shared task. We also give an overview of the various approaches adopted by the participating teams, and present the evaluation results.

Problem

• Develop methods for grammatical error correction
• Define task and evaluate correction metrics
• Compare approaches from participating teams

Innovation

• Grammatical error correction task definition
• Evaluation metric and scorer description
• Overview of participant approaches