🤖 AI Summary
This work addresses the challenge of simultaneously achieving real-time performance and high accuracy in detecting prompt-based attacks—such as jailbreaking and prompt injection—in low-latency production environments. To this end, the authors propose a lightweight, general-purpose large language model (LLM)-based security adjudicator architecture. The system employs a structured reasoning pipeline comprising intent decomposition, safety signal verification, harm assessment, and self-reflection to enable efficient threat detection. Deployed as a centralized protective service within a public-sector chatbot in Singapore, this approach demonstrates for the first time that a lightweight general-purpose LLM (e.g., gemini-2.0-flash-lite-001) can meet stringent low-latency security requirements under real-world production constraints. Additionally, the study investigates hybrid multi-model mechanisms, revealing only marginal performance gains over the single lightweight model.
📝 Abstract
Prompt attacks, including jailbreaks and prompt injections, pose a critical security risk to Large Language Model (LLM) systems. In production, guardrails must mitigate these attacks under strict low-latency constraints, resulting in a deployment gap in which lightweight classifiers and rule-based systems struggle to generalize under distribution shift, while high-capacity LLM-based judges remain too slow or costly for live enforcement. In this work, we examine whether lightweight, general-purpose LLMs can reliably serve as security judges under real-world production constraints. Through careful prompt and output design, lightweight LLMs are guided through a structured reasoning process involving explicit intent decomposition, safety-signal verification, harm assessment, and self-reflection. We evaluate our method on a curated dataset combining benign queries from real-world chatbots with adversarial prompts generated via automated red teaming (ART), covering diverse and evolving patterns. Our results show that general-purpose LLMs, such as gemini-2.0-flash-lite-001, can serve as effective low-latency judges for live guardrails. This configuration is currently deployed in production as a centralized guardrail service for public service chatbots in Singapore. We additionally evaluate a Mixture-of-Models (MoM) setting to assess whether aggregating multiple LLM judges improves prompt-attack detection performance relative to single-model judges, with only modest gains observed.