GRAPE: Guided Parameter-Space Evolution for Compact Adversarial Robustness

2026-06-12

AI Summary

This work addresses the limitation of conventional adversarial training, which operates within a fixed parameter space and overlooks the impact of optimization order on model robustness. The authors propose GRAPE, a novel framework that formulates robust training as a progressive evolution in parameter space. GRAPE dynamically allocates capacity to high-stress modules guided by adversarial spectral utilization and integrates parameter-space stabilization with progressive expansion mechanisms to enable efficient training under a controlled architecture. On CIFAR-10, GRAPE achieves a PGD-20 robust accuracy of 56.94% with ResNet-18—surpassing prior methods—while reducing the number of parameters by 21.4% and incurring only a 0.9% increase in computational overhead.

Abstract

Adversarial Training (AT) improves neural network robustness, but most methods train a fixed parameter space from the start. This paper asks whether the order in which parameters become optimizable can affect the final robust solution, even when the final architecture or computation budget is controlled. We propose GRAPE, Guided Parameter-Space Evolution, a training framework for compact adversarial robustness. GRAPE combines parameter-space stabilization with progressive hidden expansion: it stabilizes robust optimization in the currently exposed space, gradually releases new optimizable dimensions, and uses an adversarial spectral utilization score to guide newly released capacity toward high-pressure modules. In contrast to fixed-structure AT, GRAPE treats robust model learning as a process of progressive parameter-space exposure and evolution. Under the standard $\ell_\infty$ threat model on CIFAR-10, with fixed-structure ResNet-18 AT as a controlled reference, GRAPE improves PGD-20 robust accuracy from 51.70% to 56.94% at a nearly matched computation budget with a FLOPs ratio of 1.009x, while reducing parameter count by about 21.4%. A sequential grow variant with the same final ResNet-18 architecture reaches 56.52% PGD-20 robust accuracy, indicating that the gain is not only due to final architecture differences but also to the parameter-space exposure path. These results suggest that guided parameter-space evolution can yield compact and robust parameter configurations under matched computation.

Problem

• Adversarial Training
• Parameter-Space Evolution
• Robustness
• Compact Models
• Optimization Order

Innovation

• parameter-space evolution
• adversarial robustness
• progressive expansion
• guided capacity allocation
• compact neural networks