What to Forget in Unlearning? Forget Set Curation for Language Models

2026-08-14

AI Summary

This study addresses the challenge of inaccurate mapping between suppression requests and target data in machine unlearning by systematically investigating the forget set curation problem and proposing the CleanSlate benchmark. This benchmark encompasses exact extraction, content-based question answering, and capability retention evaluations to quantify the impact of various data selection strategies. Empirical results indicate that while natural language and exact matching strategies yield insufficient suppression, perception-based evaluation strategies effectively eliminate target content but induce degradation in non-target knowledge and general model capabilities. These findings underscore the critical role of data selection in balancing unlearning efficacy against unintended side effects, providing essential guidance for practical deployment.

Abstract

Machine unlearning aims to remove targeted data or behaviors from a trained model without retraining from scratch. Yet most evaluations assume that the examples to forget are already known. In realistic language-model deployments, a requester may ask a model to stop reproducing a song or book without knowing which spans, documents, quotations, or near-duplicates in a trillion-token corpus support that behavior. We study this missing upstream problem, forget set curation: mapping a suppression request to the data passed to an unlearning algorithm. We introduce CleanSlate, a benchmark for verbatim output suppression over songs and books, with model-specific extraction profiles, content-grounded QA, and capability-retention evaluations. CleanSlate exposes two failure modes. Natural lexical and exact-substring curators often yield forget sets that lead to weak suppression. An evaluation-aware curator suppresses requested continuations almost completely, but causes collateral regression on non-requested content and model-dependent capability loss. These results show that practical unlearning is not only an optimization problem once a forget set is given: the data chosen for forgetting determines both what can be unlearnt and what else is damaged.

Problem

• Machine Unlearning
• Forget Set Curation
• Language Models
• Data Selection

Innovation

• Forget Set Curation
• Machine Unlearning
• CleanSlate Benchmark
• Verbatim Output Suppression
• Collateral Regression