kNNGuard: Turning LLM Hidden Activations into a Training-Free Configurable Guardrail

2026-07-02

AI Summary

This work addresses the limitations of current safety guardrails for large language models, which predominantly rely on fine-tuning and suffer from poor generalization and high inference latency. The authors propose a lightweight, training-free guardrail mechanism that leverages hidden activations from off-the-shelf large models. By combining a small set of safe and unsafe prompt examples, the method performs k-nearest neighbor classification in both intermediate activation spaces and the embedding space, fusing the resulting scores for efficient safety judgment. This approach achieves state-of-the-art performance without fine-tuning, enables rapid cross-domain adaptation through simple updates to a compact example bank, and incurs minimal overhead—requiring less than 10 seconds to construct the sample bank. Evaluated across six domains, it matches or exceeds the F1 scores of the best fine-tuned baselines while accelerating inference by over 2.7×.

Abstract

Large language models (LLMs) are increasingly deployed in domains requiring guardrails to detect unsafe, off-topic, or adversarial prompts. Existing guardrails predominately rely on fine-tuning to build classifiers, which often suffer from low generalization and high inference latency. We present kNNGuard, a training-free guardrail that utilizes the activation space of an off-the-shelf LLM. Given a small bank of 50 safe and unsafe prompts, kNNGuard extracts hidden activations and performs multi-layer kNN fusing activation-space and embedding-space scores for classification. Across six domains spanning topical and security prompts, kNNGuard achieves competitive or superior F1 compared to fine-tuned state-of-the-art guardrails while running 2.7x faster than the best comparable guardrail, and 10x faster than a fine-tuned safety classifier without gradient updates or fine-tuning. Domain adaptation requires only updating the labeled bank, which can be constructed in under 10 seconds and several orders of magnitude faster than established guardrails. We also analyze the impact of system prompts, layer selection, and integration into production LLM pipelines as a configurable, low-latency guardrail.

Problem

• guardrails
• large language models
• safety classification
• generalization
• inference latency

Innovation

• training-free guardrail
• kNN classification
• hidden activations
• domain adaptation
• low-latency safety detection