A Red-Team Study of Anthropic Fable 5 & Opus 4.8 Models

2026-06-16

AI Summary

This study presents the first large-scale evaluation of the adversarial robustness of Anthropic’s latest closed-source large language models, Fable 5 and Opus 4.8, against automated jailbreaking attacks. Leveraging the HackAgent red-teaming framework—which integrates tree-based search, static obfuscation, and adaptive iterative strategies—the authors conducted four types of automated attacks targeting 7,826 harmful intents. Attack outcomes were validated via a three-model majority voting mechanism. Results reveal that Opus 4.8 and Fable 5 were compromised on 11.5% and 6.1% of the tested intents, respectively, yielding 1,620 and 702 automatically confirmed harmful outputs spanning all ten harm categories, with most successful attacks requiring only one to two steps. The findings uncover fine-grained structural vulnerabilities obscured by aggregate metrics, demonstrating that state-of-the-art models’ security weaknesses can be efficiently exposed without human intervention.

Abstract

We evaluate the adversarial robustness of two frontier large language models (LLMs) developed by Anthropic, Fable 5 and Opus 4.8, against four families of automated jailbreak attack across 7 826 harmful intents spanning a ten-category harm taxonomy. Using the HackAgent red-teaming framework, hundreds of thousands of adversarial attempts were generated and every apparent success was independently re-adjudicated by a panel of three judge models (majority vote). Both models resist the majority of attacks, but the residual surface is larger than aggregate framing suggests: it is dominated by adaptive iterative attacks, while static obfuscation is near-fully neutralised. The strongest adaptive search (tree-of-attacks) breaks Opus 4.8 on 11.5% of intents overall, whereas Fable 5 stays in the single digits (6.1% worst-case). Aggregate rates therefore should not be read as reassurance. Even in these hardened configurations, the two models produced 1 620 (Opus 4.8) and 702 (Fable 5) panel-confirmed harmful completions spanning every harm category, located automatically, cheaply, and within the first one or two refinement steps by an attacker model with no human expert in the loop. The reasonable conclusion is that even the best, most-tested frontier models remain reliably breakable under sustained automated pressure.

Problem

• adversarial robustness
• jailbreak attacks
• large language models
• harmful completions
• red-teaming

Innovation

• red-teaming
• adversarial robustness
• automated jailbreak
• tree-of-attacks
• large language models