Your Space is My Zone: Demystifying the Security Risks of AI-Powered Applications on Pre-Trained Model Hubs

2026-06-29

AI Summary

This study addresses emerging security threats in AI application platforms stemming from weak isolation and misconfigurations, an attack surface that has not been systematically investigated. We present the first comprehensive security analysis of three major AI platforms, establishing a threat taxonomy grounded in frameworks such as OWASP. Our analysis identifies five threat categories and ten attack vectors, uncovering three novel vulnerability classes inherent to platform architectures and demonstrating how traditional security issues are uniquely amplified within the AI ecosystem. To empirically validate these risks, we introduce Insightor—a framework integrating security modeling, static and dynamic analysis, and large-scale scanning—which we applied to over 970,000 AI applications. Our evaluation revealed thousands of applications leaking credentials, hundreds vulnerable to arbitrary code execution, and dozens embedding backdoors actively exploited in the wild, culminating in full responsible disclosure.

Abstract

AI-powered Applications (AI-Apps), hosted on platforms such as Hugging Face, are democratizing access to pre-trained models through online inference and fine-tuning services. While lowering AI adoption barriers, these platforms introduce an unexplored attack surface, as AI-Apps are often developed by untrusted parties with weak isolation and misconfigured security settings. In this paper, we present the first systematic security analysis of AI-Apps across three leading platforms. To structure our investigation, we map the AI-App lifecycle to established risk taxonomies (e.g., OWASP), identifying five threat categories and ten attack vectors ranging from generic web flaws to high-impact architectural issues. Our analysis reveals critical failures including broken access control, insecure resource reuse, insufficient input validation, and sensitive data exposure. Notably, we uncover three novel architectural vulnerabilities inherent to platform design and demonstrate how traditional issues (e.g., world-readable logs) are uniquely amplified in this ecosystem. To assess real-world impact, we develop an analysis framework Insightor and apply it to over 970,000 public AI-Apps. Alarmingly, we find thousands of apps leaking credentials, hundreds containing input injection vulnerabilities that allow arbitrary code execution, and tens harboring embedded backdoors -- indicating active exploitation. We have responsibly disclosed all findings to the affected platforms and developers.

Problem

• AI-powered Applications
• pre-trained model hubs
• security risks
• attack surface
• vulnerability

Innovation

• AI-powered Applications
• security analysis
• architectural vulnerabilities
• Insightor
• pre-trained model hubs