Cascade: Hierarchical Recoverability Control for Large Language Model Unlearning

2026-09-15

AI Summary

本文提出Cascade框架，通过路径级路由、表示级压缩和解码级干预三种控制方法减少大型语言模型中目标知识的内部可识别性，有效解决敏感或版权知识移除问题。

Abstract

Large Language Model (LLM) unlearning is essential for removing sensitive or copyrighted knowledge while preserving general utility. Existing methods often leave residual knowledge in intermediate representations, which can still be recovered. To address this, we propose Cascade, a hierarchical recoverability control framework that minimizes the internal identifiability of target knowledge. Cascade combines three complementary controls: path-level routing to suppress privacy-associated activation routes, representation-level compression to reduce geometric separability, and decoding-level intervention to limit residual recovery. Experiments on TOFU, MUSE-News, and WMDP, including robustness tests with query reformulation and extraction-style prompts, show that Cascade effectively reduces recoverability while maintaining stable model utility.

Problem

• Large Language Model
• unlearning
• sensitive knowledge
• residual knowledge
• recoverability

Innovation

• Hierarchical Recoverability Control
• Large Language Model Unlearning
• Path-level Routing
• Representation-level Compression
• Decoding-level Intervention