Minimal, Local, Causal Explanations for Jailbreak Success in Large Language Models

2026-04-30

AI Summary

Existing approaches struggle to provide local, causal explanations for how jailbreak prompts compromise large language models, relying instead on global conceptual directions with limited generalizability. This work proposes LOCA, the first method capable of delivering local causal explanations for individual successful jailbreak instances. By applying causal interventions to a model’s intermediate representations and integrating interpretability analysis with minimal intervention set search, LOCA identifies key interpretable modifications that restore the model’s refusal behavior against harmful requests. Experiments on Gemma and Llama family models demonstrate that LOCA effectively blocks jailbreak attempts with an average of only six interpretable modifications—substantially outperforming existing methods, which frequently fail even after twenty such changes.

Abstract

Safety trained large language models (LLMs) can often be induced to answer harmful requests through jailbreak prompts. Because we lack a robust understanding of why LLMs are susceptible to jailbreaks, future frontier models operating more autonomously in higher-stakes settings may similarly be vulnerable to such attacks. Prior work has studied jailbreak success by examining the model's intermediate representations, identifying directions in this space that causally encode concepts like harmfulness and refusal. Then, they globally explain all jailbreak attacks as attempting to reduce or strengthen these concepts (e.g., reduce harmfulness). However, different jailbreak strategies may succeed by strengthening or suppressing different intermediate concepts, and the same jailbreak strategy may not work for different harmful request categories (e.g., violence vs. cyberattack); thus, we seek to give a local explanation -- i.e., why did this specific jailbreak succeed? To address this gap, we introduce LOCA, a method that gives Local, CAusal explanations of jailbreak success by identifying a minimal set of interpretable, intermediate representation changes that causally induce model refusal on an otherwise successful jailbreak request. We evaluate LOCA on harmful original-jailbreak pairs from a large jailbreak benchmark across Gemma and Llama chat models, comparing against prior methods adapted to this setting. LOCA can successfully induce refusal by making, on average, six interpretable changes; prior work routinely fails to achieve refusal even after 20 changes. LOCA is a step toward mechanistic, local explanations of jailbreak success in LLMs. Code to be released.

Problem

• jailbreak
• large language models
• causal explanation
• local explanation
• safety

Innovation

• local explanation
• causal intervention
• jailbreak detection
• interpretable representations
• LLM safety