The BatchNorm Illusion: Diagnosing Normalization Artifacts in Machine Unlearning Evaluation

2026-09-08

AI Summary

研究发现并诊断了BatchNorm架构中机器遗忘评估的未记录混淆问题，通过权重保留固定点操作方法区分测量失败与编码器失败。

Abstract

Approximate machine unlearning aims to remove the influence of specific training data from a trained model without retraining from scratch. We identify a previously undocumented confound in how unlearning is evaluated on BatchNorm-based architectures: a single forward pass over retain data, an operation that modifies no weight, can deterministically rewrite the model's normalization state and reverse the apparent surface-metric forgetting. We formalize this operation as a weight-preserving fixed-point operator and prove that any pre-versus-post gap it induces is provably attributable to BN running statistics rather than to any modification the unlearning method made to the weights. This attribution claim cleanly separates measurement failure (BN artifact) from encoder failure (residual weight-encoded information, recently documented in concurrent work), and the same operator framework yields a unique decomposition of linear-probe elevation into BN-measurement-bias and encoder-geometry components. Empirically, the artifact reverses headline forget accuracy by up to 78 pp across nine evaluated methods on standard benchmarks; an attacker with as few as 10 unlabeled images recovers most of the masked accuracy; and a strict GroupNorm control reduces the artifact to zero across all methods. The tested membership-inference attacks change little under recalibration, locating the observed evaluation failure in forget accuracy and linear probing.

Problem

• BatchNorm
• machine unlearning
• normalization artifacts

Innovation

• BatchNorm Artifact
• Weight-Preserving Fixed-Point Operator
• Normalization State Rewriting
• Machine Unlearning Evaluation