BetaEdit: Null-Space Constrained Sequential Model Editing

2026-05-09

AI Summary

This work addresses a critical limitation in existing null-space-based model editing methods, which suffer from knowledge leakage during sequential edits due to imperfect null-space approximations, leading to significant performance degradation. The study elucidates the underlying mechanism of this leakage and introduces an enhanced framework that integrates history-aware updates into the null-space constraint paradigm. By systematically embedding historical edit information, the proposed approach effectively mitigates knowledge leakage and substantially improves editing stability. Extensive experiments across three prominent large language models and two standard benchmarks demonstrate that the method consistently outperforms current state-of-the-art techniques in large-scale sequential editing tasks, achieving superior edit accuracy while better preserving the model’s general capabilities.

Abstract

Null-space-based methods have garnered considerable attention in model editing by constraining updates to the null space of the pre-existing knowledge representation, thereby preserving the model's original behavior. However, in practice these methods rely on an approximate null space--leading to knowledge leakage--and further suffer from severe performance degradation during sequential editing. Recent work shows that history-aware editing strategies can empirically mitigate this decline, yet the underlying reason remains unclear. In this paper, we first expose the knowledge leakage inherent in existing null-space approaches and then analyze why history-aware updates effectively preserve both editing performance and general capabilities during long-horizon editing. Building on these insights, we propose BetaEdit, a refined framework that effectively controls the knowledge leakage and integrates history-aware updates into the null-space paradigm. Extensive experiments on three large language models across two standard benchmarks show that BetaEdit consistently outperforms prior methods in the challenging regime of massive-scale sequential editing. Code is available at: https://github.com/lbq8942/BetaEdit.

Problem

• model editing
• null-space
• knowledge leakage
• sequential editing
• performance degradation

Innovation

• null-space editing
• knowledge leakage
• sequential model editing
• history-aware update
• BetaEdit