H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking

2025-02-18

AI Summary

This work exposes a fundamental vulnerability in the safety judgment mechanisms of Large Reasoning Models (LRMs) that rely on Chain-of-Thought (CoT) reasoning: attackers can bypass safety guardrails using malicious queries disguised as educational prompts (the Malicious-Educator benchmark). To address this, the authors propose H-CoT—the first general, transferable adversarial attack paradigm for CoT-based LRMs—requiring no fine-tuning or external intervention. H-CoT dynamically constructs adversarial prompts by reverse-engineering the model’s internal CoT reasoning process to hijack its safety decision logic. Experiments demonstrate that H-CoT reduces OpenAI o1’s safety refusal rate from 98% to under 2%, and successfully elicits criminal strategy generation from DeepSeek-R1 and Gemini 2.0 Flash Thinking. These results reveal systemic safety design flaws in commercially deployed LRMs operating under the “reasoning-as-a-service” paradigm.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements

Abstract

Large Reasoning Models (LRMs) have recently extended their powerful reasoning capabilities to safety checks-using chain-of-thought reasoning to decide whether a request should be answered. While this new approach offers a promising route for balancing model utility and safety, its robustness remains underexplored. To address this gap, we introduce Malicious-Educator, a benchmark that disguises extremely dangerous or malicious requests beneath seemingly legitimate educational prompts. Our experiments reveal severe security flaws in popular commercial-grade LRMs, including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking. For instance, although OpenAI's o1 model initially maintains a high refusal rate of about 98%, subsequent model updates significantly compromise its safety; and attackers can easily extract criminal strategies from DeepSeek-R1 and Gemini 2.0 Flash Thinking without any additional tricks. To further highlight these vulnerabilities, we propose Hijacking Chain-of-Thought (H-CoT), a universal and transferable attack method that leverages the model's own displayed intermediate reasoning to jailbreak its safety reasoning mechanism. Under H-CoT, refusal rates sharply decline-dropping from 98% to below 2%-and, in some instances, even transform initially cautious tones into ones that are willing to provide harmful content. We hope these findings underscore the urgent need for more robust safety mechanisms to preserve the benefits of advanced reasoning capabilities without compromising ethical standards.

Problem

• Exposes security flaws in Large Reasoning Models
• Introduces Malicious-Educator benchmark for testing
• Proposes Hijacking Chain-of-Thought attack method

Innovation

• Hijacks Chain-of-Thought reasoning
• Utilizes Malicious-Educator benchmark
• Exposes vulnerabilities in LRMs