Towards Localized and Disentangled Knowledge Editing for Multimodal Large Language Models

2026-05-28

AI Summary

This work addresses the limited generalization of existing knowledge editing methods for multimodal large language models to logically related queries and their tendency to inadvertently alter unrelated yet semantically or visually associated information. The authors propose LDKE, a novel framework that formally characterizes the problems of causal misalignment and feature entanglement in knowledge editing. LDKE integrates localized editing with an input decoupling mechanism to precisely update target knowledge at specific model layers. It introduces a fast localization module to identify critical layers and employs a decoupled classifier to route inputs, ensuring edits affect only relevant knowledge. Experiments demonstrate that LDKE significantly enhances editing generalization across multiple benchmarks and multimodal large language models while effectively preserving the fidelity of unrelated knowledge.

Abstract

Existing methods in Multimodal Knowledge Editing (MKE) have advanced the ability to correct outdated or inaccurate knowledge in Multimodal Large Language Models (MLLMs). However, they exhibit a critical limitation: while effectively modifying target factual pairs, they fail to generalize edits to logically related queries and often cause unintended alterations to unrelated but visually or semantically linked information. We identify and formalize two underlying failure modes causing this issue: Causal Misalignment, which confines edits to the specific sample, and Feature Entanglement, which causes unintended alterations to coupled but irrelevant information. To address these issues, we propose Localized and Disentangled Knowledge Editing (LDKE), a new framework that achieves precise and generalized editing by localizing fact-specific model layers and disentangling target-relevant inputs from irrelevant ones. Our approach introduces a Fast Localization module to identify and update critical layers efficiently, along with a Disentanglement Classifier that routes inputs appropriately to preserve unrelated knowledge. Extensive experiments across various benchmarks and MLLMs demonstrate that LDKE achieves superior performance in propagating edits to related contexts while maintaining high locality.

Problem

• Multimodal Knowledge Editing
• Causal Misalignment
• Feature Entanglement
• Knowledge Localization
• Edit Generalization

Innovation

• Localized Editing
• Disentangled Representation
• Multimodal Knowledge Editing
• Causal Alignment
• Feature Disentanglement