SHARE: Shared Memory-Aware Open-Domain Long-Term Dialogue Dataset Constructed from Movie Script

2024-10-28

arXiv.org

AI Summary

To address insufficient coherence and user engagement in open-domain long-horizon dialogues, this paper proposes a shared-memory enhancement paradigm. First, we introduce SHARE, the first structured shared-memory dataset explicitly designed for long-horizon dialogue, constructed from movie scripts to model character identities and events explicitly while implicitly identifying transferable shared memories. Second, we formally define and annotate both explicit and implicit shared memories—marking the first systematic effort—and propose the EPISODE framework to dynamically model, retrieve, and maintain memory consistency across dialogue turns. Third, empirical evaluation demonstrates that integrating shared memory significantly improves dialogue coherence and user engagement; EPISODE achieves superior performance over existing baselines in memory recall accuracy and response consistency. The SHARE dataset is publicly released to support future research.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Conversational AI/Dialog Systems
• Machine Learning: Life-Long and Continual Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Economics, Online Markets and Human Computation: The sharing economy

Abstract

Shared memories between two individuals strengthen their bond and are crucial for facilitating their ongoing conversations. This study aims to make long-term dialogue more engaging by leveraging these shared memories. To this end, we introduce a new long-term dialogue dataset named SHARE, constructed from movie scripts, which are a rich source of shared memories among various relationships. Our dialogue dataset contains the summaries of persona information and events of two individuals, as explicitly revealed in their conversation, along with implicitly extractable shared memories. We also introduce EPISODE, a long-term dialogue framework based on SHARE that utilizes shared experiences between individuals. Through experiments using SHARE, we demonstrate that shared memories between two individuals make long-term dialogues more engaging and sustainable, and that EPISODE effectively manages shared memories during dialogue. Our new dataset is publicly available at https://anonymous.4open.science/r/SHARE-AA1E/SHARE.json.

Problem

• Enhancing long-term dialogue engagement using shared memories
• Creating a dataset from movie scripts for shared memory analysis
• Developing a framework to manage shared memories in dialogues

Innovation

• Constructs SHARE dataset from movie scripts
• Utilizes shared memories for engaging dialogues
• Introduces EPISODE framework for memory management