ReLiK: Retrieve and LinK, Fast and Accurate Entity Linking and Relation Extraction on an Academic Budget

2024-07-31

Annual Meeting of the Association for Computational Linguistics

AI Summary

To address the inefficiency, slow inference, and high resource consumption in jointly modeling entity linking (EL) and relation extraction (RE) for large-scale NLP tasks, this paper proposes cIE, a retrieval-reading dual-stage joint framework. Methodologically, cIE introduces a novel unified input representation that jointly encodes candidate entities and relations, enabling end-to-end alignment via a single forward pass through a shared reader; it further incorporates a contextualized candidate injection mechanism and leverages lightweight fine-tuning of pretrained language models for efficient joint decoding. Contributions include: (1) achieving cross-domain EL+RE joint SOTA under limited academic computational resources for the first time; (2) attaining state-of-the-art performance on multiple in-domain and cross-domain benchmarks; (3) accelerating inference by up to 40× compared to prior joint models; and (4) setting new records across multiple key metrics on the cIE task.

Technology Category

• Natural Language Processing: Information Extraction
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Entity Linking (EL) and Relation Extraction (RE) are fundamental tasks in Natural Language Processing, serving as critical components in a wide range of applications. In this paper, we propose ReLiK, a Retriever-Reader architecture for both EL and RE, where, given an input text, the Retriever module undertakes the identification of candidate entities or relations that could potentially appear within the text. Subsequently, the Reader module is tasked to discern the pertinent retrieved entities or relations and establish their alignment with the corresponding textual spans. Notably, we put forward an innovative input representation that incorporates the candidate entities or relations alongside the text, making it possible to link entities or extract relations in a single forward pass and to fully leverage pre-trained language models contextualization capabilities, in contrast with previous Retriever-Reader-based methods, which require a forward pass for each candidate. Our formulation of EL and RE achieves state-of-the-art performance in both in-domain and out-of-domain benchmarks while using academic budget training and with up to 40x inference speed compared to competitors. Finally, we show how our architecture can be used seamlessly for Information Extraction (cIE), i.e. EL + RE, and setting a new state of the art by employing a shared Reader that simultaneously extracts entities and relations.

Problem

• Develops ReLiK for fast, accurate Entity Linking and Relation Extraction
• Introduces single-pass Retriever-Reader architecture leveraging pre-trained models
• Achieves state-of-the-art performance with 40x faster inference speed

Innovation

• Retriever-Reader architecture for EL and RE
• Single forward pass with candidate integration
• Shared Reader for simultaneous entity and relation extraction