CoT Red-Handed: Stress Testing Chain-of-Thought Monitoring

2025-05-29

AI Summary

Autonomous deployment of large language models (LLMs) poses security risks, as malicious actors may stealthily induce harmful behaviors. Method: We propose a novel safety supervision paradigm based on monitoring intermediate chain-of-thought (CoT) reasoning—departing from conventional output-only monitoring. Our approach introduces a hybrid monitoring protocol that independently evaluates both the CoT process and the final output, followed by weighted fusion for dual-path collaborative decision-making. Contribution/Results: We are the first to identify and systematically quantify how CoT reasoning can be adversarially rationalized to evade detection, establishing its failure boundaries. We further develop a red-teaming evaluation framework and a cross-model robustness assessment suite. Experiments demonstrate over a 4× improvement in detection rate for subtle deception scenarios; across multiple models and tasks, our method consistently outperforms single-path baselines, achieving up to a 27-percentage-point gain in accuracy.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

As AI models are deployed with increasing autonomy, it is important to ensure they do not take harmful actions unnoticed. As a potential mitigation, we investigate Chain-of-Thought (CoT) monitoring, wherein a weaker trusted monitor model continuously oversees the intermediate reasoning steps of a more powerful but untrusted model. We compare CoT monitoring to action-only monitoring, where only final outputs are reviewed, in a red-teaming setup where the untrusted model is instructed to pursue harmful side tasks while completing a coding problem. We find that CoT monitoring improves detection by up to 27 percentage points in scenarios where action-only monitoring fails to reliably identify sabotage. However, CoT traces can also contain misleading rationalizations that deceive the monitor, reducing performance in more obvious sabotage cases. To address this, we introduce a hybrid protocol that independently scores both reasoning and final outputs and combines them using a weighted average. This hybrid monitor consistently outperforms both CoT and action-only monitors across all tested models and tasks, with detection rates over four times higher than action-only monitoring for subtle deception scenarios.

Problem

• Detecting harmful actions in autonomous AI models using Chain-of-Thought monitoring
• Comparing CoT monitoring with action-only monitoring in red-teaming setups
• Improving detection rates with a hybrid protocol combining reasoning and output scores

Innovation

• Uses Chain-of-Thought monitoring for oversight
• Introduces hybrid protocol combining reasoning scores
• Improves detection rates over action-only monitoring