Comparative Insights on Adversarial Machine Learning from Industry and Academia: A User-Study Approach

2026-02-04

AI Summary

The widespread adoption of generative AI has intensified security risks associated with adversarial machine learning (AML), yet a significant gap persists between industry and academia in awareness and educational approaches. This study innovatively integrates user research with hands-on pedagogy by conducting practitioner surveys and student-facing Capture-the-Flag (CTF) challenges to systematically compare how these two groups perceive AML threats. Leveraging natural language processing and generative AI techniques, the project also develops a data poisoning attack demonstration to contextualize these risks. Findings indicate that cybersecurity education substantially heightens attention to AML issues, and the CTF-based approach effectively fosters student engagement. These insights offer empirical support and concrete recommendations for reforming machine learning curricula to better integrate security education.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Security and Privacy: Security and privacy of machine learning and AI applications
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

An exponential growth of Machine Learning and its Generative AI applications brings with it significant security challenges, often referred to as Adversarial Machine Learning (AML). In this paper, we conducted two comprehensive studies to explore the perspectives of industry professionals and students on different AML vulnerabilities and their educational strategies. In our first study, we conducted an online survey with professionals revealing a notable correlation between cybersecurity education and concern for AML threats. For our second study, we developed two CTF challenges that implement Natural Language Processing and Generative AI concepts and demonstrate a poisoning attack on the training data set. The effectiveness of these challenges was evaluated by surveying undergraduate and graduate students at Carnegie Mellon University, finding that a CTF-based approach effectively engages interest in AML threats. Based on the responses of the participants in our research, we provide detailed recommendations emphasizing the critical need for integrated security education within the ML curriculum.

Problem

• Adversarial Machine Learning
• Security Education
• Generative AI
• Machine Learning Security
• User Study

Innovation

• Adversarial Machine Learning
• CTF-based education
• User study
• Security education
• Generative AI security