SERE: Structural Example Retrieval for Enhancing LLMs in Event Causality Identification

2026-05-05

AI Summary

This work addresses the tendency of large language models (LLMs) to generate causal hallucinations—over-predicting causal relationships—in event causality identification tasks. To mitigate this bias, the authors propose SERE, a few-shot reasoning framework guided by structured example retrieval. SERE innovatively integrates three complementary strategies: a ConceptNet-based conceptual path metric, a dependency parse tree edit distance to assess structural similarity, and an LLM-driven causal pattern filter to select high-quality in-context examples. Experimental results demonstrate that SERE significantly improves accuracy across multiple event causality identification benchmarks while substantially reducing over-prediction, thereby validating its effectiveness and robustness.

Abstract

Event Causality Identification (ECI) requires models to determine whether a given pair of events in a context exhibits a causal relationship. While Large Language Models (LLMs) have demonstrated strong performance across various NLP tasks, their effectiveness in ECI remains limited due to biases in causal reasoning, often leading to overprediction of causal relationships (causal hallucination). To mitigate these issues and enhance LLM performance in ECI, we propose SERE, a structural example retrieval framework that leverages LLMs' few-shot learning capabilities. SERE introduces an innovative retrieval mechanism based on three structural concepts: (i) Conceptual Path Metric, which measures the conceptual relationship between events using edit distance in ConceptNet; (ii) Syntactic Metric, which quantifies structural similarity through tree edit distance on syntactic trees; and (iii) Causal Pattern Filtering, which filters examples based on predefined causal structures using LLMs. By integrating these structural retrieval strategies, SERE selects more relevant examples to guide LLMs in causal reasoning, mitigating bias and improving accuracy in ECI tasks. Extensive experiments on multiple ECI datasets validate the effectiveness of SERE. The source code is publicly available at https://github.com/DMIRLAB-Group/SERE.

Problem

• Event Causality Identification
• Large Language Models
• Causal Hallucination
• Causal Reasoning Bias

Innovation

• Structural Example Retrieval
• Event Causality Identification
• Conceptual Path Metric
• Syntactic Tree Edit Distance
• Causal Pattern Filtering