An Empirical Study on the Security Vulnerabilities of GPTs

2025-11-28

AI Summary

This study presents the first systematic security analysis of OpenAI’s custom AI agents (GPTs), uncovering critical, architecture-wide vulnerabilities arising from their uniform cross-domain deployment. To model attack surfaces holistically, we adopt dual perspectives—platform and user—and design an empirical attack suite covering information leakage, tool misuse, and other threat dimensions. Through attack surface analysis, controlled prompt injection, and toolchain hijacking experiments, we validate multiple cross-component attack vectors, including configuration leakage, context-based privilege escalation, and plugin privilege elevation. We propose lightweight defenses: an instruction sandbox, a tool invocation whitelist, and sensitive context filtering—all rigorously evaluated in production GPT environments. Our work establishes a methodological foundation and practical benchmark for security assessment and defense of customized LLM agents.

Technology Category

• Multiagent Systems: Adversarial Agents
• Natural Language Processing: Safety and Robustness
• Philosophy and Ethics of AI: Privacy & Security

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Equipped with various tools and knowledge, GPTs, one kind of customized AI agents based on OpenAI's large language models, have illustrated great potential in many fields, such as writing, research, and programming. Today, the number of GPTs has reached three millions, with the range of specific expert domains becoming increasingly diverse. However, given the consistent framework shared among these LLM agent applications, systemic security vulnerabilities may exist and remain underexplored. To fill this gap, we present an empirical study on the security vulnerabilities of GPTs. Building upon prior research on LLM security, we first adopt a platform-user perspective to conduct a comprehensive attack surface analysis across different system components. Then, we design a systematic and multidimensional attack suite with the explicit objectives of information leakage and tool misuse based on the attack surface analysis, thereby concretely demonstrating the security vulnerabilities that various components of GPT-based systems face. Finally, we accordingly propose defense mechanisms to address the aforementioned security vulnerabilities. By increasing the awareness of these vulnerabilities and offering critical insights into their implications, this study seeks to facilitate the secure and responsible application of GPTs while contributing to developing robust defense mechanisms that protect users and systems against malicious attacks.

Problem

• Identifies security vulnerabilities in GPT-based AI agents
• Analyzes attack surfaces for information leakage and tool misuse
• Proposes defenses to protect systems from malicious attacks

Innovation

• Adopt platform-user perspective for attack surface analysis
• Design systematic multidimensional attack suite for vulnerabilities
• Propose defense mechanisms to address security vulnerabilities