Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models

2024-08-14

arXiv.org

AI Summary

Large language models (LLMs) frequently suffer performance degradation during continual knowledge editing due to unintended interference with non-target knowledge, primarily caused by representational overlap between old and newly edited knowledge. Method: This work systematically investigates the failure mechanism from the perspective of knowledge superposition. We theoretically prove that superposition is the root cause of editing interference—lossless editing is achievable only in its absence—and establish a quantitative relationship between superposition strength and interference magnitude. Using closed-form solutions of linear associative memory, analytical decomposition of interference terms, and statistical modeling of cross-model knowledge distributions (via kurtosis, heavy-tailedness, and scaling laws), we empirically validate the universality and statistical regularities of superposition across diverse LLMs. Contribution: This is the first study to formally characterize superposition as the core cause of editing failure in continual settings. We provide theoretical guarantees, quantitative metrics, and empirical evidence across models, along with open-sourced code and datasets to ensure reproducibility.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Knowledge editing aims to update outdated or incorrect knowledge in large language models (LLMs). However, current knowledge editing methods have limited scalability for lifelong editing. This study explores the fundamental reason why knowledge editing fails in lifelong editing. We begin with the closed-form solution derived from linear associative memory, which underpins state-of-the-art knowledge editing methods. We extend the solution from single editing to lifelong editing, and through rigorous mathematical derivation, identify an interference term in the final solution, suggesting that editing knowledge may impact irrelevant knowledge. Further analysis of the interference term reveals a close relationship with superposition between knowledge representations. When knowledge superposition does not exist in language models, the interference term vanishes, allowing for lossless knowledge editing. Experiments across numerous language models reveal that knowledge superposition is universal, exhibiting high kurtosis, zero mean, and heavy-tailed distributions with clear scaling laws. Ultimately, by combining theory and experiments, we demonstrate that knowledge superposition is the fundamental reason for the failure of lifelong editing. Moreover, this is the first study to investigate knowledge editing from the perspective of superposition and provides a comprehensive observation of superposition across numerous real-world language models. Code available at https://github.com/ChenhuiHu/knowledge_in_superposition.

Problem

• Continuous Learning
• Knowledge Overlap
• Model Performance Degradation

Innovation

• Knowledge Overlap
• Language Models
• Continuous Learning