GONE: Structural Knowledge Unlearning via Neighborhood-Expanded Distribution Shaping

2026-02-21

arXiv.org

AI Summary

本文针对大语言模型中知识遗忘问题，提出GONE基准及NEDS框架，有效处理结构化数据中的直接事实移除、推理泄露和灾难性遗忘。

Abstract

Unlearning knowledge is a pressing and challenging task in Large Language Models (LLMs) because of their unprecedented capability to memorize and digest training data at scale, raising more significant issues regarding safety, privacy, and intellectual property. However, existing works, including parameter editing, fine-tuning, and distillation-based methods, are all focused on flat sentence-level data but overlook the relational, multi-hop, and reasoned knowledge in naturally structured data. In response to this gap, this paper introduces Graph Oblivion and Node Erasure (GONE), a benchmark for evaluating knowledge unlearning over structured knowledge graph (KG) facts in LLMs. This KG-based benchmark enables the disentanglement of three effects of unlearning: direct fact removal, reasoning-based leakage, and catastrophic forgetting. In addition, Neighborhood-Expanded Distribution Shaping (NEDS), a novel unlearning framework, is designed to leverage graph connectivity and identify anchor correlated neighbors, enforcing a precise decision boundary between the forgotten fact and its semantic neighborhood. Evaluations on LLaMA-3-8B and Mistral-7B across multiple knowledge editing and unlearning methods showcase NEDS's superior performance (1.000 on unlearning efficacy and 0.839 on locality) on GONE and other benchmarks. Code is available at https://anonymous.4open.science/r/GONE-4679/.

Problem

• knowledge unlearning
• large language models
• structured data
• relational knowledge
• reasoning-based leakage

Innovation

• Graph Oblivion and Node Erasure (GONE)
• Neighborhood-Expanded Distribution Shaping (NEDS)
• knowledge unlearning
• structured knowledge graph (KG)