A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition

2025-02-25

AI Summary

Zero-shot named entity recognition (NER) suffers from insufficient contextual modeling and misleading demonstrations. To address these challenges, we propose the Collaborative Multi-Agent System (CMAS), which decomposes NER into two subtasks: entity span detection and type-specific feature extraction. CMAS introduces a novel four-agent architecture—Self-Annotation Agent, Type-Feature Extractor, Demonstration Discriminator, and Joint Predictor—that explicitly models semantic associations between entities and context, incorporates a self-reflection mechanism to quantify demonstration utility, and employs dynamic demonstration filtering to mitigate reasoning biases in large language models (LLMs) under zero-shot settings. Evaluated on six general-domain and domain-specific zero-shot NER benchmarks, CMAS achieves substantial improvements over strong baselines. It is compatible with diverse LLM backbones and maintains robust generalization in few-shot settings.

Technology Category

• Multiagent Systems: Modeling other Agents
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Zero-shot named entity recognition (NER) aims to develop entity recognition systems from unannotated text corpora. This task presents substantial challenges due to minimal human intervention. Recent work has adapted large language models (LLMs) for zero-shot NER by crafting specialized prompt templates. It advances model self-learning abilities by incorporating self-annotated demonstrations. However, two important challenges persist: (i) Correlations between contexts surrounding entities are overlooked, leading to wrong type predictions or entity omissions. (ii) The indiscriminate use of task demonstrations, retrieved through shallow similarity-based strategies, severely misleads LLMs during inference. In this paper, we introduce the cooperative multi-agent system (CMAS), a novel framework for zero-shot NER that uses the collective intelligence of multiple agents to address the challenges outlined above. CMAS has four main agents: (i) a self-annotator, (ii) a type-related feature (TRF) extractor, (iii) a demonstration discriminator, and (iv) an overall predictor. To explicitly capture correlations between contexts surrounding entities, CMAS reformulates NER into two subtasks: recognizing named entities and identifying entity type-related features within the target sentence. To enable controllable utilization of demonstrations, a demonstration discriminator is established to incorporate the self-reflection mechanism, automatically evaluating helpfulness scores for the target sentence. Experimental results show that CMAS significantly improves zero-shot NER performance across six benchmarks, including both domain-specific and general-domain scenarios. Furthermore, CMAS demonstrates its effectiveness in few-shot settings and with various LLM backbones.

Problem

• Develops zero-shot NER from unannotated texts.
• Addresses context correlation and demonstration misuse.
• Improves NER with cooperative multi-agent system.

Innovation

• Cooperative multi-agent system for NER
• Self-annotator and type feature extractor
• Demonstration discriminator with self-reflection