Adversarial ML Problems Are Getting Harder to Solve and to Evaluate

2025-02-04

AI Summary

The rise of large language models (LLMs) is triggering a fundamental degeneration in adversarial machine learning (AML): problem definitions are becoming increasingly ambiguous, solution complexity is markedly increasing, and evaluation outcomes are growing unreliable. Method: To systematically expose this trend, the paper introduces the first “definition–solution–evaluation” tripartite deterioration analytical framework. Through conceptual clarification, comparative analysis of research paradigms, and critical examination of evaluation methodologies—without relying on specific algorithms or empirical experiments—it delivers a theoretical diagnosis of AML’s evolutionary impasse in the LLM era. Contribution/Results: The core contribution is a prescriptive critical framework that reveals a severe paucity of substantive progress, persistent erosion of reproducibility and verifiability, and an imminent risk of prolonged stagnation in AML over the next decade. This work serves as an urgent warning to the community regarding foundational methodological fragility in LLM-era adversarial research.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

In the past decade, considerable research effort has been devoted to securing machine learning (ML) models that operate in adversarial settings. Yet, progress has been slow even for simple"toy"problems (e.g., robustness to small adversarial perturbations) and is often hindered by non-rigorous evaluations. Today, adversarial ML research has shifted towards studying larger, general-purpose language models. In this position paper, we argue that the situation is now even worse: in the era of LLMs, the field of adversarial ML studies problems that are (1) less clearly defined, (2) harder to solve, and (3) even more challenging to evaluate. As a result, we caution that yet another decade of work on adversarial ML may fail to produce meaningful progress.

Problem

• Adversarial ML problems increasingly complex
• Evaluation methods lack rigor and clarity
• Progress in adversarial ML remains slow

Innovation

• adversarial ML challenges
• LLMs complexity
• non-rigorous evaluations