Beyond Surface-Level Detection: Towards Cognitive-Driven Defense Against Jailbreak Attacks via Meta-Operations Reasoning

2025-08-04

AI Summary

Existing LLM jailbreaking defenses rely heavily on shallow pattern matching, resulting in poor generalization to unseen attack strategies. To address this, we propose a cognition-driven meta-operation defense framework that models implicit malicious operational logic in prompts via hierarchical structured reasoning. Our method integrates entropy-guided reinforcement learning (EG-GRPO) to enable adaptive identification and response to novel jailbreaking tactics. By synergistically combining supervised fine-tuning with human-like cognitive mechanisms—balancing global contextual awareness and local analytical precision—the framework achieves interpretable, evolution-aware defense. Experiments across mainstream jailbreaking benchmarks demonstrate state-of-the-art performance, superior robustness against zero-shot and out-of-distribution attacks, and significantly enhanced generalization compared to rule-based or discriminative-model baselines. This work establishes a new paradigm for LLM security: one that is both cognitively grounded and inherently adaptable.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Defending large language models (LLMs) against jailbreak attacks is essential for their safe and reliable deployment. Existing defenses often rely on shallow pattern matching, which struggles to generalize to novel and unseen attack strategies. To address this challenge, we propose the Cognitive-Driven Defense (CDD) framework, which targets the underlying structure of jailbreak prompts by applying meta-operations, defined as basic manipulations that conceal harmful intent.CDD emulates human cognitive reasoning through a structured reasoning chain. It begins with a global perception of the prompt and follows with a localized analysis to uncover hidden manipulations. By applying supervised fine-tuning on this structured chain, the model learns to identify and reason about known manipulation patterns. To enhance generalization to unseen threats, an entropy-guided reinforcement learning algorithm (EG-GRPO) is introduced to encourage exploration of new types and variants of meta-operations. Experiments demonstrate that CDD can achieve state-of-the-art defense performance and exhibit strong generalization to unseen jailbreak attacks.

Problem

• Defending LLMs against novel jailbreak attack strategies
• Identifying hidden manipulations in prompts via cognitive reasoning
• Enhancing generalization to unseen threats with reinforcement learning

Innovation

• Cognitive-Driven Defense via meta-operations reasoning
• Structured reasoning chain for hidden manipulation detection
• Entropy-guided RL for exploring unseen meta-operations