Synthetic Worlds for Temporal Evaluation and Knowledge Updating in LLMs

2026-08-31

AI Summary

本文针对LLM知识随时间过时的问题，提出了一种基于模拟的框架ParallelEvents和训练框架Synapse，以实现模型参数的有效更新与知识整合。

Abstract

Large language models (LLMs) rely on static pretraining corpora, causing their knowledge to become outdated over time. Existing approaches for evaluating knowledge edits either suffer from rapid contamination or rely on counterfactual edits that conflict with rigid existing knowledge. In this work, we propose a synthetic, simulation-driven framework for studying knowledge insertion in LLMs. We introduce {\sc ParallelEvents}, a benchmark of fictional yet realistic future worlds that generates coherent event trajectories for controlled evaluation, avoiding contamination while preserving consistency. Building on this dataset, we develop {\sc Synapse}, a training framework that uses model-generated data to update model parameters via mid-training and instruction tuning. This synthetic pipeline enables scalable knowledge integration without costly human-curated data. Empirically, {\sc Synapse} outperforms existing methods by 14.23\%, demonstrating that simulation-based synthetic training leads to robust and coherent knowledge insertions.

Problem

• large language models
• static pretraining corpora
• knowledge outdated
• evaluation of knowledge edits
• rapid contamination

Innovation

• synthetic framework
• ParallelEvents
• Synapse
• knowledge insertion