A Scalable Framework for Automated NER Annotation Correction in Low-Resource Languages

2026-09-16

AI Summary

本文提出一个多步骤框架，通过自训练和双阈值机制提高NER数据集标注质量，解决低资源语言中NER标注噪声问题。

Abstract

Poor quality or noisy annotations in Named Entity Recognition (NER), as in any other NLP task, make it challenging to achieve state-of-the-art performance. In this paper, we present a multi-step framework to enhance the annotation quality of NER datasets by employing automated techniques. We propose a frequency-based iterative approach that leverages self-training and a dual-threshold mechanism to enhance inference confidence. Experimental evaluations on different NER datasets demonstrate significant improvements in NER performance with respect to the original datasets. This work further explores the potential of generative Large Language Models (LLMs) to perform NER for low-resource languages.

Problem

• Named Entity Recognition
• Low-Resource Languages
• Annotation Quality

Innovation

• multi-step framework
• frequency-based iterative approach
• self-training
• dual-threshold mechanism
• generative Large Language Models (LLMs)