Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs

2025-03-03

AI Summary

Existing large language model (LLM) knowledge editing methods exhibit insufficient sensitivity to relational changes, resulting in poor edit locality and persistent irrelevant facts—thereby undermining model reliability. To address this, we propose a fine-grained, neuron-level knowledge editing framework that enables, for the first time, relation-aware localization and intervention at the single-neuron level. Our method integrates causal tracing–enhanced neuron sensitivity analysis with a relation-guided knowledge localization mechanism to precisely identify and modify individual neurons in feed-forward layers that exhibit strong functional association with the target fact. This approach transcends coarse-grained, module-level editing paradigms. Evaluated across multiple benchmarks, it achieves a 12.7% improvement in edit accuracy and reduces irrelevant fact retention by 41%, significantly enhancing both controllability and reliability of knowledge updates.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Action, Change, and Causality
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Knowledge editing aims to update outdated information in Large Language Models (LLMs). A representative line of study is locate-then-edit methods, which typically employ causal tracing to identify the modules responsible for recalling factual knowledge about entities. However, we find these methods are often sensitive only to changes in the subject entity, leaving them less effective at adapting to changes in relations. This limitation results in poor editing locality, which can lead to the persistence of irrelevant or inaccurate facts, ultimately compromising the reliability of LLMs. We believe this issue arises from the insufficient precision of knowledge localization. To address this, we propose a Fine-grained Neuron-level Knowledge Editing (FiNE) method that enhances editing locality without affecting overall success rates. By precisely identifying and modifying specific neurons within feed-forward networks, FiNE significantly improves knowledge localization and editing. Quantitative experiments demonstrate that FiNE efficiently achieves better overall performance compared to existing techniques, providing new insights into the localization and modification of knowledge within LLMs.

Problem

• Improves precision in localizing knowledge within LLMs
• Enhances editing locality without reducing success rates
• Addresses limitations in adapting to changes in relations

Innovation

• Fine-grained neuron-level knowledge editing technique
• Precise identification and modification of specific neurons
• Enhanced editing locality without affecting success rates