EL4NER: Ensemble Learning for Named Entity Recognition via Multiple Small-Parameter Large Language Models

2025-05-29

AI Summary

Contextual learning (ICL) for named entity recognition (NER) using large language models (LLMs) incurs high computational costs, data privacy risks, and deployment challenges due to reliance on massive closed- or open-source LLMs. Method: This paper proposes the first multi-stage ensemble ICL framework tailored for small-parameter open-source LLMs (e.g., Phi-3, TinyLlama). It introduces (1) a task-decomposed ensemble paradigm that separates NER into span boundary detection and type classification; (2) a span-level sentence similarity retrieval algorithm specifically designed for NER; and (3) a self-verification mechanism that dynamically filters low-confidence predictions to suppress ensemble noise. Results: Experiments across multiple NER benchmarks demonstrate that our approach outperforms most ICL-based methods leveraging closed-source LLMs and achieves state-of-the-art performance among existing ICL approaches in several settings—while drastically improving parameter efficiency and practical deployability.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

In-Context Learning (ICL) technique based on Large Language Models (LLMs) has gained prominence in Named Entity Recognition (NER) tasks for its lower computing resource consumption, less manual labeling overhead, and stronger generalizability. Nevertheless, most ICL-based NER methods depend on large-parameter LLMs: the open-source models demand substantial computational resources for deployment and inference, while the closed-source ones incur high API costs, raise data-privacy concerns, and hinder community collaboration. To address this question, we propose an Ensemble Learning Method for Named Entity Recognition (EL4NER), which aims at aggregating the ICL outputs of multiple open-source, small-parameter LLMs to enhance overall performance in NER tasks at less deployment and inference cost. Specifically, our method comprises three key components. First, we design a task decomposition-based pipeline that facilitates deep, multi-stage ensemble learning. Second, we introduce a novel span-level sentence similarity algorithm to establish an ICL demonstration retrieval mechanism better suited for NER tasks. Third, we incorporate a self-validation mechanism to mitigate the noise introduced during the ensemble process. We evaluated EL4NER on multiple widely adopted NER datasets from diverse domains. Our experimental results indicate that EL4NER surpasses most closed-source, large-parameter LLM-based methods at a lower parameter cost and even attains state-of-the-art (SOTA) performance among ICL-based methods on certain datasets. These results show the parameter efficiency of EL4NER and underscore the feasibility of employing open-source, small-parameter LLMs within the ICL paradigm for NER tasks.

Problem

• Reducing computational and API costs in NER using small-parameter LLMs
• Enhancing NER performance via ensemble learning with multiple LLMs
• Improving ICL demonstration retrieval for NER tasks

Innovation

• Ensemble learning with small-parameter LLMs
• Task decomposition for multi-stage ensemble
• Span-level similarity for ICL retrieval