Generalizable Multimodal Large Language Model Editing via Invariant Trajectory Learning

2026-01-27

AI Summary

This work addresses the challenge that multimodal large language models often suffer from causal underfitting or overfitting under out-of-distribution (OOD) cross-modal prompts due to rigid parameter-to-output mappings, hindering reliable, localized, and generalizable knowledge editing. The authors reformulate this issue as an OOD generalization problem and propose ODEdit, a novel framework that introduces invariant learning to multimodal editing for the first time. By disentangling semantic shifts from factual changes, ODEdit learns invariant causal editing trajectories that remain stable under environmental perturbations. The method incorporates a triple OOD risk objective and total variation regularization to effectively suppress spurious correlations. Both theoretical analysis and extensive experiments demonstrate that ODEdit significantly outperforms existing approaches in terms of reliability, locality, and cross-modal generalization.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Knowledge editing emerges as a crucial technique for efficiently correcting incorrect or outdated knowledge in large language models (LLM). Existing editing methods rely on a rigid mapping from parameter or module modifications to output, which causes the generalization limitation in Multimodal LLM (MLLM). In this paper, we reformulate MLLM editing as an out-of-distribution (OOD) generalization problem, where the goal is to discern semantic shift with factual shift and thus achieve robust editing among diverse cross-modal prompting. The key challenge of this OOD problem lies in identifying invariant causal trajectories that generalize accurately while suppressing spurious correlations. To address it, we propose ODEdit, a plug-and-play invariant learning based framework that optimizes the tripartite OOD risk objective to simultaneously enhance editing reliability, locality, and generality.We further introduce an edit trajectory invariant learning method, which integrates a total variation penalty into the risk minimization objective to stabilize edit trajectories against environmental variations. Theoretical analysis and extensive experiments demonstrate the effectiveness of ODEdit.

Problem

• Out-of-Distribution Generalization
• Multimodal Large Language Model
• Knowledge Editing
• Invariant Trajectories
• Causal Reasoning

Innovation

• out-of-distribution generalization
• invariant learning
• multimodal large language model editing
• edit trajectory
• spurious correlation