Past Meets Present: Creating Historical Analogy with Large Language Models

2024-09-23

Annual Meeting of the Association for Computational Linguistics

AI Summary

This paper addresses the historical analogy retrieval problem—automatically identifying semantically and contextually appropriate historical events analogous to contemporary ones to support decision-making and interpretive understanding. Methodologically, it proposes a dual-path framework integrating large language model (LLM)-driven cross-temporal semantic alignment, retrieval-augmented generation (RAG), and prompt engineering, augmented by a novel self-reflective reasoning mechanism to mitigate hallucination and historical stereotyping biases. Contributions include: (i) the first multidimensional automatic evaluation framework for historical analogies, jointly assessing accuracy, relevance, and historical plausibility—validated via both human annotation and automated metrics; and (ii) empirical results demonstrating statistically significant improvements in analogy quality, thereby establishing the feasibility and superiority of LLMs for historical analogy tasks.

Technology Category

• Cognitive Modeling & Cognitive Systems: Analogy
• Knowledge Representation and Reasoning: Argumentation
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Historical analogies, which compare known past events with contemporary but unfamiliar events, are important abilities that help people make decisions and understand the world. However, research in applied history suggests that people have difficulty finding appropriate analogies. And previous studies in the AI community have also overlooked historical analogies. To fill this gap, in this paper, we focus on the historical analogy acquisition task, which aims to acquire analogous historical events for a given event. We explore retrieval and generation methods for acquiring historical analogies based on different large language models (LLMs). Furthermore, we propose a self-reflection method to mitigate hallucinations and stereotypes when LLMs generate historical analogies. Through human evaluations and our specially designed automatic multi-dimensional assessment, we find that LLMs generally have a good potential for historical analogies. And the performance of the models can be further improved by using our self-reflection method.

Problem

• Identifying analogous historical events for unfamiliar contemporary situations
• Addressing challenges in retrieving and generating accurate historical analogies
• Mitigating hallucinations and stereotypes in LLM-generated historical analogies

Innovation

• Retrieval and generation methods using LLMs
• Self-reflection to reduce hallucinations and stereotypes
• Multi-dimensional assessment for historical analogy evaluation