What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics

2026-06-23

AI Summary

Current alignment techniques for large language models remain vulnerable to jailbreaking attacks, and existing defenses are largely confined to input or output layers, lacking insight into internal representations of harmful intent. This work addresses this gap by analyzing token-level prediction entropy dynamics in frozen intermediate layers of LLMs through a logit lens, revealing for the first time that jailbreaking signals predominantly emerge in middle layers rather than at the output layer. The authors introduce an entropy-based dynamic feature grounded in monotonic rank trends, which enables effective jailbreak detection without fine-tuning or additional training. Evaluated across diverse architectures—including Llama, Qwen, and Gemma—and multiple adversarial benchmarks, the method demonstrates high discriminative power, cross-model consistency, and practical applicability.

Abstract

Jailbreak attacks reveal a persistent weakness in aligned Large Language Models: carefully crafted prompts can elicit policy-violating responses despite safety training. While most defenses operate at the prompt or output level, it remains unclear how harmful intent is encoded within the model's internal representations. We investigate this question by analyzing token-level predictive entropy trajectories across layers of a frozen LLM using the logit lens. We find that static aggregate statistics of prompt-level entropy (e.g., mean, variance) carry little discriminative signal, whereas features capturing how entropy evolves across token positions, such as monotonic rank-based trend scores, are substantially more informative. Importantly, this signal is not uniform across model depth: it is concentrated in intermediate layers and degrades at the final layer, indicating that jailbreak-relevant structure is most pronounced in mid-network representations rather than at the output head. Across multiple models (Llama, Qwen, Gemma) and adversarial benchmarks, these entropy dynamics provide architecture-consistent separation without additional training. Together, our findings show that jailbreak behavior is reflected in structured intermediate uncertainty dynamics, clarifying both which entropy-derived features encode harmful intent and where in the network that signal is most pronounced.

Problem

• jailbreak attacks
• Large Language Models
• internal representations
• predictive entropy
• safety alignment

Innovation

• entropy dynamics
• jailbreak detection
• intermediate layers
• logit lens
• predictive uncertainty