Generative AI for Named Entity Recognition in Low-Resource Language Nepali

2025-03-12

AI Summary

This work addresses named entity recognition (NER) for the low-resource language Nepali. We conduct the first systematic evaluation of mainstream large language models (LLMs) on generative zero-shot and few-shot NER without fine-tuning. To enable rigorous assessment, we construct a manually annotated Nepali NER test set and empirically evaluate both open- and closed-source LLMs using zero-shot, few-shot, and chain-of-thought prompting strategies. Results demonstrate that LLMs possess foundational NER capability, achieving up to 72.4% accuracy; moreover, our lightweight prompt engineering approach significantly enhances adaptability to low-resource settings. This study fills a critical gap by establishing the first publicly available benchmark for Nepali NER, providing a reproducible methodological framework and the inaugural performance benchmark for generative NER in low-resource languages.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Generative Artificial Intelligence (GenAI), particularly Large Language Models (LLMs), has significantly advanced Natural Language Processing (NLP) tasks, such as Named Entity Recognition (NER), which involves identifying entities like person, location, and organization names in text. LLMs are especially promising for low-resource languages due to their ability to learn from limited data. However, the performance of GenAI models for Nepali, a low-resource language, has not been thoroughly evaluated. This paper investigates the application of state-of-the-art LLMs for Nepali NER, conducting experiments with various prompting techniques to assess their effectiveness. Our results provide insights into the challenges and opportunities of using LLMs for NER in low-resource settings and offer valuable contributions to the advancement of NLP research in languages like Nepali.

Problem

• Evaluating GenAI for Nepali Named Entity Recognition
• Assessing LLMs' effectiveness with limited Nepali data
• Exploring challenges and opportunities in low-resource NER

Innovation

• Uses Large Language Models for Nepali NER
• Explores prompting techniques for low-resource languages
• Evaluates GenAI effectiveness in limited data scenarios