Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Models

2026-06-15

AI Summary

Current large reasoning models lack intrinsic safety mechanisms and rely heavily on extensive human-annotated data for safety alignment. This work proposes a novel approach that requires no external annotations by eliciting the model’s inherent “latent safety awareness,” enabling it to adaptively trigger safety analysis and guidance while preserving general capabilities. Specifically, the method integrates a safety-label triggering mechanism into supervised fine-tuning (SFT) and leverages direct preference optimization (DPO) to enhance the accuracy and stability of safe responses. Evaluated on DeepSeek-R1-Distill-Llama-8B, the approach reduces the success rates of harmful queries and jailbreak attacks by 24.65% and 36.72% on average, respectively, with negligible impact on general performance.

Abstract

While Large Reasoning Models (LRMs) excel at complex tasks, they remain highly vulnerable to sophisticated jailbreaks and direct harmful queries. To address this vulnerability, prior works depend heavily on external manual data annotation for safety alignment. However, we observe that LRMs can inherently identify safety risks when being re-presented with original queries alongside their own reasoning trajectories -- a capability we term Latent Safety Awareness. To leverage this safety awareness, we first employ Supervised Fine-Tuning (SFT) to explicitly induce safe tags to trigger safety analysis and guidance following the initial reasoning content for unsafe queries, while preserving standard responses for general queries to ensure adaptive triggering. Subsequently, we apply Direct Preference Optimization (DPO) to further enhance the correctness and stability of the safety analysis and guidance. Notably, responses required for both training stages are entirely generated by models being optimized. With (Safe Trigger) SFT and DPO, experimental results demonstrate significant safety enhancement. For example, the Attack Success Rate (ASR) of DeepSeek-R1-Distill-Llama-8B, on average, drops 24.65% and 36.72% on harmful and jailbreak benchmarks, respectively. Finally, our Safe Trigger method exerts almost no negative impact on general performance or user experience.

Problem

• Large Reasoning Models
• jailbreak attacks
• harmful queries
• safety alignment
• Latent Safety Awareness

Innovation

• Latent Safety Awareness
• Safe Trigger
• Supervised Fine-Tuning (SFT)
• Direct Preference Optimization (DPO)
• Large Reasoning Models