Mechanistic Circuit-Based Knowledge Editing in Large Language Models

2026-04-07

AI Summary

This work addresses the “reasoning gap” commonly observed in large language models during dynamic knowledge updating—where models retain edited facts but fail to correctly apply them in multi-step reasoning. To bridge this gap, the authors propose MCircKE, a novel framework that introduces mechanistic causal circuit analysis into knowledge editing for the first time. By identifying and precisely adjusting the parameters of causal circuits responsible for specific reasoning tasks, MCircKE jointly localizes factual storage and reasoning pathways, enabling their coordinated editing. The method implements a “map-and-adapt” editing pipeline and achieves substantial improvements on the MQuAKE-3K benchmark in multi-hop reasoning scenarios, effectively overcoming the limitations of conventional approaches that modify isolated facts without considering their downstream inferential use.

Abstract

Deploying Large Language Models (LLMs) in real-world dynamic environments raises the challenge of updating their pre-trained knowledge. While existing knowledge editing methods can reliably patch isolated facts, they frequently suffer from a "Reasoning Gap", where the model recalls the edited fact but fails to utilize it in multi-step reasoning chains. To bridge this gap, we introduce MCircKE (\underline{M}echanistic \underline{Circ}uit-based \underline{K}nowledge \underline{E}diting), a novel framework that enables a precise "map-and-adapt" editing procedure. MCircKE first identifies the causal circuits responsible for a specific reasoning task, capturing both the storage of the fact and the routing of its logical consequences. It then surgically update parameters exclusively within this mapped circuit. Extensive experiments on the MQuAKE-3K benchmark demonstrate the effectiveness of the proposed method for multi-hop reasoning in knowledge editing.

Problem

• knowledge editing
• reasoning gap
• large language models
• multi-hop reasoning

Innovation

• Mechanistic Circuit
• Knowledge Editing
• Reasoning Gap
• Causal Circuit
• Multi-hop Reasoning