Neural Variability Enhances Artificial Network Robustness

2026-06-11

AI Summary

This work addresses the limited robustness of artificial neural networks under adversarial attacks and natural image corruptions, a challenge often exacerbated by the neglect of structured noise in neural activations. The authors propose a biologically inspired local noise mechanism that models structured noise by analyzing the covariance structure of activations induced by clean and perturbed inputs. Relying solely on local information, this approach is the first to systematically reveal how structured noise differentially enhances robustness across perturbation types. Experimental results demonstrate that the proposed strategy significantly improves model robustness against natural corruptions, and notably, the noise structures learned under adversarial attacks exhibit strong generalization to other attack variants.

Abstract

Neural responses in cortex exhibit substantial trial-to-trial variability in response to repeated stimuli, while peripheral sensory neurons respond far more consistently, leading many to wonder whether stochasticity may carry meaning. Existing work has argued that noise and signal correlations may be optimized for discrimination in animals, whereas artificial neural network (ANN) studies have shown similar benefits of noise in machine learning tasks, although most ANN work has neglected the effects of correlations. Here we investigate whether correlated noise improves the robustness of artificial neural networks to adversarial attacks and naturalistic image modifications. Using the covariance of activations under modified versus clean inputs, we find that structured noise may significantly improve network robustness. Robustness to naturalistic image modifications benefits most from structure, but this structure transfers poorly across modification types. In contrast, noise structure from adversarial attacks can generalize to other kinds of attacks. These results suggest that structured noise in ANN activations generally improves robustness, establishing a biologically plausible strategy for creating robust artificial neural networks that only relies on local information.

Problem

• neural variability
• robustness
• adversarial attacks
• naturalistic image modifications
• correlated noise

Innovation

• structured noise
• neural variability
• adversarial robustness
• activation covariance
• biologically plausible learning