Is nasty noise actually harder than malicious noise?

2025-11-12

AI Summary

This work investigates the learnability of Boolean functions under two adversarial noise models: malicious noise and nasty noise. We first establish that, in the distribution-free setting, these models are computationally equivalent; however, under a fixed (known) distribution, we construct—under standard cryptographic assumptions—instances exhibiting an arbitrarily large separation in noise tolerance. To bridge this gap, we propose the “ignore contradictory examples” algorithmic paradigm, formally modeling ICE-style algorithms and proving that their optimal loss when converting between the two noise models is at most a constant factor of 2—a bound shown to be tight. Our results unify computational learning theory, adversarial modeling, and cryptographic analysis, yielding fundamental advances in the theoretical foundations of robust learning under adversarial noise.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Game Theory and Economic Paradigms: Adversarial Learning
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Applications of cryptography
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

We consider the relative abilities and limitations of computationally efficient algorithms for learning in the presence of noise, under two well-studied and challenging adversarial noise models for learning Boolean functions: malicious noise, in which an adversary can arbitrarily corrupt a random subset of examples given to the learner; and nasty noise, in which an adversary can arbitrarily corrupt an adversarially chosen subset of examples given to the learner. We consider both the distribution-independent and fixed-distribution settings. Our main results highlight a dramatic difference between these two settings: For distribution-independent learning, we prove a strong equivalence between the two noise models: If a class ${cal C}$ of functions is efficiently learnable in the presence of $eta$-rate malicious noise, then it is also efficiently learnable in the presence of $eta$-rate nasty noise. In sharp contrast, for the fixed-distribution setting we show an arbitrarily large separation: Under a standard cryptographic assumption, for any arbitrarily large value $r$ there exists a concept class for which there is a ratio of $r$ between the rate $eta_{malicious}$ of malicious noise that polynomial-time learning algorithms can tolerate, versus the rate $eta_{nasty}$ of nasty noise that such learning algorithms can tolerate. To offset the negative result for the fixed-distribution setting, we define a broad and natural class of algorithms, namely those that ignore contradictory examples (ICE). We show that for these algorithms, malicious noise and nasty noise are equivalent up to a factor of two in the noise rate: Any efficient ICE learner that succeeds with $eta$-rate malicious noise can be converted to an efficient learner that succeeds with $eta/2$-rate nasty noise. We further show that the above factor of two is necessary, again under a standard cryptographic assumption.

Problem

• Comparing malicious and nasty noise models in Boolean function learning
• Analyzing noise tolerance differences between distribution settings
• Evaluating algorithms that ignore contradictory examples under noise

Innovation

• Compares malicious and nasty noise learning models
• Establishes equivalence in distribution-independent learning settings
• Introduces ICE algorithms for noise tolerance equivalence