🤖 AI Summary
Downsampling in CNNs violates the Nyquist–Shannon sampling theorem, causing aliasing and spectral leakage—degrading model robustness to distribution shifts and adversarial attacks. To address this, we propose ASAP Pooling: the first plug-and-play downsampling layer jointly mitigating aliasing and spectral leakage, rigorously grounded in signal processing theory. ASAP integrates an anti-aliasing filter with a leakage-suppressing mechanism to enable distortion-free frequency-domain downsampling, while preserving computational efficiency and architectural compatibility. Merely replacing standard pooling with ASAP—requiring only fine-tuning of the filter coefficients—yields substantial gains in native robustness: robust accuracy on ImageNet-C improves markedly; defense against high- and low-resolution adversarial examples strengthens significantly; and clean-data accuracy is maintained—or even slightly improved—without trade-offs.
📝 Abstract
Convolutional neural networks encode images through a sequence of convolutions, normalizations and non-linearities as well as downsampling operations into potentially strong semantic embeddings. Yet, previous work showed that even slight mistakes during sampling, leading to aliasing, can be directly attributed to the networks' lack in robustness. To address such issues and facilitate simpler and faster adversarial training, [12] recently proposed FLC pooling, a method for provably alias-free downsampling - in theory. In this work, we conduct a further analysis through the lens of signal processing and find that such current pooling methods, which address aliasing in the frequency domain, are still prone to spectral leakage artifacts. Hence, we propose aliasing and spectral artifact-free pooling, short ASAP. While only introducing a few modifications to FLC pooling, networks using ASAP as downsampling method exhibit higher native robustness against common corruptions, a property that FLC pooling was missing. ASAP also increases native robustness against adversarial attacks on high and low resolution data while maintaining similar clean accuracy or even outperforming the baseline.
