LASH: Adaptive Semantic Hybridization for Black-Box Jailbreaking of Large Language Models

2026-05-20

AI Summary

Existing black-box jailbreaking attacks suffer from limited generalization across diverse models and harmful categories due to reliance on a single strategy. This work proposes LASH, a novel framework that achieves adaptive semantic fusion of heterogeneous jailbreaking strategies for the first time. LASH employs a derivative-free genetic optimizer guided by a two-stage fitness function—combining keyword-based refusal detection and LLM-as-a-judge scoring—to dynamically select an optimal subset of seed prompts and tune their mixing weights, thereby generating highly effective jailbreak prompts. Evaluated on JailbreakBench, LASH attains average attack success rates of 84.5% (keyword-based) and 74.5% (two-stage) using only 30 queries, substantially outperforming five state-of-the-art methods while maintaining robustness under three distinct defense mechanisms.

Abstract

Jailbreak attacks expose a persistent gap between the intended safety behavior of aligned large language models and their behavior under adversarial prompting. Existing automated methods are increasingly effective but each commits to a single attack family (e.g., one refinement loop, one tree search, one mutation space, or one strategy library) and no single family dominates: the best-performing method shifts across target models and harm categories, suggesting complementary strengths that per-prompt composition could exploit. We introduce LASH (LLM Adaptive Semantic Hybridization), a black-box framework that treats outputs from multiple base attacks as reusable seed prompts and adaptively composes them for each target request. Given a seed pool, LASH searches over seed subsets and softmax-normalized mixture weights; a composition module synthesizes a single candidate prompt, and a derivative-free genetic optimizer updates the weights using black-box target feedback and a two-stage fitness function combining keyword-based refusal detection with LLM-judge scoring. On JailbreakBench, which contains 100 harmful prompts across 10 categories, we evaluate LASH on six common target models. LASH achieves an average attack success rate of 84.5% under keyword-based evaluation and 74.5% under two-stage evaluation, where responses are first filtered for refusals and then scored by an LLM judge for whether they substantively fulfill the original harmful request. LASH outperforms five state-of-the-art baselines on both metrics with only 30 mean target queries. LASH also remains competitive under three defense mechanisms and induces more success-like internal representations. These results suggest that adaptive composition across heterogeneous jailbreak strategies is a promising direction for black-box red-teaming.

Problem

• jailbreak attacks
• large language models
• black-box attacks
• adaptive composition
• safety alignment

Innovation

• adaptive composition
• black-box jailbreaking
• semantic hybridization
• genetic optimization
• heterogeneous attack strategies