Adaptive Schema-aware Event Extraction with Retrieval-Augmented Generation

2025-05-13

AI Summary

Event extraction (EE) in real-world scenarios requires dynamic matching and precise identification from vast candidate schemas, yet existing methods are constrained by rigid schema assumptions and the absence of comprehensive, joint evaluation benchmarks. Method: We propose MD-SEE—the first schema-aware, multi-dimensional evaluation benchmark covering domain diversity, structural complexity, and cross-lingual transfer—and introduce a synergistic framework integrating semantic schema rewriting with retrieval-augmented generation. This framework unifies dense retrieval, LLM instruction fine-tuning, and structured output constraints to mitigate schema hallucination and overcome context-length limitations. Contribution/Results: Experiments demonstrate substantial improvements in cross-domain accuracy on MD-SEE, with exceptional robustness in long-tail and low-resource settings. Our approach achieves, for the first time, end-to-end adaptive event structure generation without predefined schema constraints.

Technology Category

• Natural Language Processing: Information Extraction
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Event extraction (EE) is a fundamental task in natural language processing (NLP) that involves identifying and extracting event information from unstructured text. Effective EE in real-world scenarios requires two key steps: selecting appropriate schemas from hundreds of candidates and executing the extraction process. Existing research exhibits two critical gaps: (1) the rigid schema fixation in existing pipeline systems, and (2) the absence of benchmarks for evaluating joint schema matching and extraction. Although large language models (LLMs) offer potential solutions, their schema hallucination tendencies and context window limitations pose challenges for practical deployment. In response, we propose Adaptive Schema-aware Event Extraction (ASEE), a novel paradigm combining schema paraphrasing with schema retrieval-augmented generation. ASEE adeptly retrieves paraphrased schemas and accurately generates targeted structures. To facilitate rigorous evaluation, we construct the Multi-Dimensional Schema-aware Event Extraction (MD-SEE) benchmark, which systematically consolidates 12 datasets across diverse domains, complexity levels, and language settings. Extensive evaluations on MD-SEE show that our proposed ASEE demonstrates strong adaptability across various scenarios, significantly improving the accuracy of event extraction.

Problem

• Adaptive schema selection for event extraction
• Overcoming LLM schema hallucination limitations
• Lack of benchmarks for joint schema matching

Innovation

• Adaptive schema paraphrasing and retrieval-augmented generation
• Multi-Dimensional Schema-aware Event Extraction benchmark
• Combines schema retrieval with targeted structure generation