LOKI: Memory-Free Null-Space Constrained Lifelong Knowledge Editing

2026-06-17

AI Summary

This work addresses the limitations of existing continual knowledge editing methods—namely, insufficient flexibility due to fixed-layer updates, catastrophic forgetting, and reliance on historical data and extensive preprocessing—by proposing a memory-free, efficient editing framework. The approach integrates a dynamic layer selection strategy with null-space gradient projection based on the Hilbert-Schmidt Independence Criterion (HSIC) to constrain update directions without accessing past data, thereby preserving previously acquired knowledge. Experimental results demonstrate that the proposed method substantially enhances model adaptability and stability in continual learning settings, achieving up to a 14% improvement in average accuracy over state-of-the-art baselines across multiple benchmarks.

Abstract

Lifelong knowledge editing aims to efficiently and sequentially update language models over time, as new knowledge becomes available or when the model makes mistakes, while preserving acceptable performance on past knowledge. One unresolved challenge is that existing methods modify a fixed set of layers for all new knowledge samples, reducing flexibility and increasing catastrophic forgetting. Another is requiring access to previous knowledge and extensive pre-processing to obtain data statistics. To address these challenges, we introduce LOKI, a novel approach that uses dynamic layer selection based on the Hilbert-Schmidt Independence Criterion and projects gradient updates onto the null-space of the model weights, bypassing the requirement for previous knowledge access. We show that LOKI achieves superior performance to existing approaches across a wide variety of experiments, achieving up to a 14\% improvement in average accuracy.

Problem

• lifelong knowledge editing
• catastrophic forgetting
• memory-free
• null-space constrained
• dynamic layer selection

Innovation

• dynamic layer selection
• null-space projection
• Hilbert-Schmidt Independence Criterion
• memory-free editing
• lifelong knowledge editing