Latent Knowledge Scalpel: Precise and Massive Knowledge Editing for Large Language Models

2025-07-31

AI Summary

Large language models (LLMs) often generate factually incorrect or biased outputs due to outdated or erroneous knowledge embedded in their pretraining data; existing knowledge editing methods struggle to simultaneously support large-scale concurrent edits and preserve model generalization. This paper proposes a latent-space-guided knowledge editing framework that employs a lightweight hypernetwork to directly modulate internal entity representations—enabling precise localization and substitution of entity-specific knowledge without updating the backbone parameters. Our method supports synchronous editing of up to ten thousand factual assertions, achieving, for the first time, natural-language-style, high-fidelity, and highly scalable knowledge updates. Experiments on Llama-2 and Mistral demonstrate that after editing 10,000 facts, the models retain downstream task performance and general capabilities with no statistically significant degradation, substantially outperforming state-of-the-art editing approaches.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Large Language Models (LLMs) often retain inaccurate or outdated information from pre-training, leading to incorrect predictions or biased outputs during inference. While existing model editing methods can address this challenge, they struggle with editing large amounts of factual information simultaneously and may compromise the general capabilities of the models. In this paper, our empirical study demonstrates that it is feasible to edit the internal representations of LLMs and replace the entities in a manner similar to editing natural language inputs. Based on this insight, we introduce the Latent Knowledge Scalpel (LKS), an LLM editor that manipulates the latent knowledge of specific entities via a lightweight hypernetwork to enable precise and large-scale editing. Experiments conducted on Llama-2 and Mistral show even with the number of simultaneous edits reaching 10,000, LKS effectively performs knowledge editing while preserving the general abilities of the edited LLMs. Code is available at: https://github.com/Linuxin-xxx/LKS.

Problem

• Editing inaccurate or outdated knowledge in LLMs
• Enabling large-scale simultaneous factual edits
• Preserving model general capabilities during editing

Innovation

• Edits internal representations of LLMs
• Uses lightweight hypernetwork for manipulation
• Enables large-scale precise knowledge editing