BARRIER: Bounded Activation Regions for Robust Information Erasure

2026-05-15

AI Summary

Existing machine unlearning methods often degrade retained knowledge when erasing target concepts and lack theoretical guarantees, leading to significant performance drops. This work proposes a novel unlearning mechanism grounded in the geometric structure of activation spaces: shifting the intervention focus from static weights to the dynamic distribution of hidden-layer activations, performing unlearning updates within a bounded hypercube, and— for the first time—formalizing knowledge retention as an optimization objective with probabilistic tail bounds. By integrating singular value decomposition with interval arithmetic to construct activation boundaries, the method rigorously preserves non-target representations while aggressively removing specified concepts. Experiments demonstrate that the proposed framework achieves state-of-the-art performance on both classification and diffusion models, effectively balancing thorough forgetting with comprehensive knowledge retention.

Abstract

Machine unlearning has reached a critical bottleneck. As traditional weight-space interventions focus primarily on erasing targeted concepts, they often fail to prevent the unintended suppression of other significant representations. This leads to substantial collateral damage, with essential knowledge being forgotten, because these methods lack formal mathematical guarantees for the preservation of neutral concepts. To avoid degradation, they are frequently forced into conservative updates. We propose BARRIER (Bounded Activation Regions for Robust Information Erasure), a paradigm-shifting framework that shifts the locus of intervention from static model weights to the dynamic geometry of hidden-layer activations. Unlike existing methods, BARRIER employs Interval Arithmetic (IA) on SVD-based projections of the activation space to encapsulate the specific target region within a bounding hypercube. By driving unlearning updates exclusively within this forget interval and mathematically bounding the model response on the complement, we ensure rigorous protection of the retain distribution. This geometric construction transforms the preservation of knowledge from an empirical heuristic into a formal optimization target with a probabilistic tail bound on functional drift. Crucially, this stability permits highly aggressive unlearning updates within the forget region. Empirical evaluations demonstrate that BARRIER matches state-of-the-art trade-offs across classifiers and diffusion models, maximizing targeted concept erasure while safeguarding the integrity of all other representations. Our code is available at https://github.com/OneAndZero24/BARRIER.

Problem

• machine unlearning
• concept erasure
• representation preservation
• collateral damage
• knowledge retention

Innovation

• machine unlearning
• activation geometry
• interval arithmetic
• knowledge preservation
• bounded updates