From AI-Generated Content to Agentic Action: Security and Safety Threats in Generative AI

2026-05-15

AI Summary

This study systematically investigates the security challenges arising as generative AI transitions from content generation to performing real-world actions, introducing a novel tripartite threat taxonomy encompassing content-level, model-level, and agent-level risks. Through integrated threat modeling, evaluation of technical countermeasures—including detection, watermarking, alignment techniques, and agent-specific safeguards—and analysis of governance structures, the work reveals a pervasive gap between the rapid expansion of attack surfaces and the current state of defensive capabilities. Most existing technical solutions remain contingent on nascent institutional coordination mechanisms that have yet to mature. The research underscores the necessity for parallel evolution of technical and governance approaches and highlights the critical importance of cross-layer collaborative defense strategies to effectively mitigate emerging threats.

Abstract

Generative AI systems are increasingly used not only to produce content but also to retrieve data, invoke tools, and execute actions. This work examines the security and safety implications of that shift across content-level, model-level, and agentic threats. We analyze how attacker access requirements, system autonomy, and the scope of potential harm change as models move from generating artifacts to executing operations through tool chains and external APIs. We then assess technical countermeasures including detection, watermarking, alignment, and emerging agentic safeguards, and show that several depend on forms of institutional coordination that current governance arrangements do not yet provide. Across the cases examined, capability deployment and attack-surface expansion repeatedly outpace defensive responses as systems move from generating content to executing real-world actions.

Problem

• Generative AI
• Security threats
• Agentic action
• Safety risks
• Attack surface

Innovation

• agentic AI
• generative AI security
• tool-use threats
• AI governance
• attack surface expansion