PVDetector: Detecting Prompt Injection Attacks on Purpose-Specific LLM Agents through Policy-Violation Concept Analysis

2026-07-14

AI Summary

This work addresses the vulnerability of large language model (LLM) agents to prompt injection attacks when handling out-of-scope or unauthorized requests, a challenge for which existing detection methods offer limited efficacy. The authors propose a training-free detection mechanism that, for the first time, identifies and leverages an intrinsic “Policy Violation” (PV) semantic concept embedded in the model’s hidden activation space. By constructing contrastive pairs of violating and compliant prompts offline, the method extracts a PV direction; during inference, it detects attacks by measuring the alignment between the input’s hidden states and this direction. Experiments demonstrate that the approach achieves false negative rates below 1% across multiple mainstream LLMs and datasets, substantially outperforming state-of-the-art baselines while incurring minimal computational overhead.

Abstract

Large language models (LLMs) are increasingly deployed as purpose-specific agents to handle domain-specific tasks such as customer service and code generation. These agents are expected to comply with not only generic safety guardrails but also purpose-specific restrictions tailored to their designated roles. Such additional restrictions enlarge the attack surface, particularly to prompt injection (PI) attacks. To defend against such attacks, existing detection methods primarily rely on analyzing input-output patterns, yet yield limited effectiveness. To address this limitation, we turn to analyzing the hidden activation space and discover that LLMs inherently retain latent policy-violation (PV) concepts when prompted with requests beyond their designated purpose. Particularly, PV concepts capture the semantics of conflicts between user queries and predefined restrictions, implicitly reflecting LLMs' intrinsic awareness of recognizing policy violations. Building on this insight, we propose PVDetector, a training-free framework that detects PI attacks during LLM inference by measuring hidden-state alignment with PV concepts, which are derived offline from the contrastive pairs of policy-violating and policy-compliant prompts. Experiments across multiple LLMs and datasets show that PVDetector achieves <1\% false negative rate with minimal auxiliary overhead, consistently outperforming state-of-the-art methods. Our code is available at https://github.com/Claresigle/PVDetector .

Problem

• prompt injection
• large language models
• policy violation
• purpose-specific agents
• safety guardrails

Innovation

• Prompt Injection
• Policy Violation
• Hidden Activation Analysis
• Training-Free Detection
• Purpose-Specific LLM Agents