K-Edit: Language Model Editing with Contextual Knowledge Awareness

2025-02-15

AI Summary

To address context inconsistency arising from single-point edits during knowledge updates in large language models (LLMs), this paper proposes a knowledge graph (KG)-guided contextual editing method. The core innovation lies in the first integration of KG structural consistency into model editing, introducing a “contextual editing” mechanism: multi-hop semantic associations are identified via KG embeddings; gradient projection is applied for parameter updates, while edge-propagation-driven contextual generation and multi-hop consistency constraints jointly optimize edited outputs. This approach ensures high single-edit accuracy while synchronously correcting associated knowledge, significantly improving multi-hop question answering performance. It supports up to thousands of concurrent edits, demonstrating strong scalability. Crucially, post-editing generalization capability remains intact—no degradation in downstream task performance is observed.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

As the world changes, we need to be able to update our models and correct false information without costly retraining. Knowledge-based model editing enables precise modifications to the weights of large language models in order to modify the information encoded within. Recent approaches have seen success in enabling recall of edited information for thousands of edits at once. However, these approaches fail to produce edits that account for associated contextual information. We present K-Edit, an effective approach to generating contextually consistent knowledge edits. By using knowledge graphs, which maintain contextual consistency when an edge is edited, we are able to generate additional extit{contextual edits} that ensure consistency of related information in the language model. Our experiments demonstrate significant improvements in multi-hop question answering while maintaining the general effectiveness and scalability of model edits.

Problem

• Update models without retraining
• Ensure contextual consistency in edits
• Improve multi-hop question answering

Innovation

• Contextual Knowledge Awareness
• Knowledge Graphs Integration
• Precision Model Editing