Beyond Local Edits: Embedding-Virtualized Knowledge for Broader Evaluation and Preservation of Model Editing

2026-02-02

AI Summary

Current evaluation methods for knowledge editing in large language models are constrained by limited datasets, making it difficult to comprehensively characterize the impact of edits on the model’s overall knowledge system. This work proposes the Embedding-Virtualized Knowledge (EVK) framework, which, for the first time, constructs virtualized knowledge regions in embedding space through controlled perturbations to quantify knowledge drift and enable broader assessment of editing effects. Building upon this, we introduce a plug-in EVK-Align module that significantly enhances knowledge retention without compromising editing accuracy. Leveraging the EVK framework, we develop EVK-Bench, a new evaluation benchmark that uncovers editing side effects overlooked by conventional metrics, thereby establishing a more comprehensive and fine-grained paradigm for evaluating knowledge editing.

Technology Category

• Knowledge Representation and Reasoning: Knowledge Engineering
• Data Mining & Knowledge Management: Knowledge Acquisition from the Web
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Knowledge editing methods for large language models are commonly evaluated using predefined benchmarks that assess edited facts together with a limited set of related or neighboring knowledge. While effective, such evaluations remain confined to finite, dataset-bounded samples, leaving the broader impact of editing on the model's knowledge system insufficiently understood. To address this gap, we introduce Embedding-Virtualized Knowledge (EVK) that characterizes model knowledge through controlled perturbations in embedding space, enabling the exploration of a substantially broader and virtualized knowledge region beyond explicit data annotations. Based on EVK, we construct an embedding-level evaluation benchmark EVK-Bench that quantifies potential knowledge drift induced by editing, revealing effects that are not captured by conventional sample-based metrics. Furthermore, we propose a plug-and-play EVK-Align module that constrains embedding-level knowledge drift during editing and can be seamlessly integrated into existing editing methods. Experiments demonstrate that our approach enables more comprehensive evaluation while significantly improving knowledge preservation without sacrificing editing accuracy.

Problem

• knowledge editing
• large language models
• knowledge preservation
• evaluation benchmark
• embedding space

Innovation

• Embedding-Virtualized Knowledge
• knowledge editing
• knowledge drift
• EVK-Bench
• EVK-Align