Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy

2025-03-26

AI Summary

This work exposes a fundamental vulnerability of safety-aligned large language models (LLMs) and multimodal LLMs (MLLMs) under out-of-distribution (OOD) harmful inputs: current alignment methods fail to generalize to OOD adversarial samples. Method: We propose JOOD—a gradient-free, model-agnostic jailbreaking framework that applies lightweight cross-modal OOD perturbations (e.g., image mixup, text style transfer, OCR noise injection, and semantic confusion) to disrupt the model’s discrimination of malicious intent and bypass safety guardrails. Contribution/Results: Leveraging uncertainty-driven attack strategies, JOOD achieves >85% success rates against strongly aligned commercial models (e.g., GPT-4, o1), significantly outperforming prior approaches. This is the first systematic study to uncover the failure mechanisms of safety alignment under OOD conditions, empirically validating its generalization limits. Our findings provide critical insights for robust safety alignment and establish a new benchmark for evaluating alignment robustness against distributional shifts.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Despite the remarkable versatility of Large Language Models (LLMs) and Multimodal LLMs (MLLMs) to generalize across both language and vision tasks, LLMs and MLLMs have shown vulnerability to jailbreaking, generating textual outputs that undermine safety, ethical, and bias standards when exposed to harmful or sensitive inputs. With the recent advancement of safety alignment via preference-tuning from human feedback, LLMs and MLLMs have been equipped with safety guardrails to yield safe, ethical, and fair responses with regard to harmful inputs. However, despite the significance of safety alignment, research on the vulnerabilities remains largely underexplored. In this paper, we investigate the unexplored vulnerability of the safety alignment, examining its ability to consistently provide safety guarantees for out-of-distribution(OOD)-ifying harmful inputs that may fall outside the aligned data distribution. Our key observation is that OOD-ifying the vanilla harmful inputs highly increases the uncertainty of the model to discern the malicious intent within the input, leading to a higher chance of being jailbroken. Exploiting this vulnerability, we propose JOOD, a new Jailbreak framework via OOD-ifying inputs beyond the safety alignment. We explore various off-the-shelf visual and textual transformation techniques for OOD-ifying the harmful inputs. Notably, we observe that even simple mixing-based techniques such as image mixup prove highly effective in increasing the uncertainty of the model, thereby facilitating the bypass of the safety alignment. Experiments across diverse jailbreak scenarios demonstrate that JOOD effectively jailbreaks recent proprietary LLMs and MLLMs such as GPT-4 and o1 with high attack success rate, which previous attack approaches have consistently struggled to jailbreak. Code is available at https://github.com/naver-ai/JOOD.

Problem

• Investigates vulnerability of LLMs to jailbreaking via out-of-distribution inputs
• Proposes JOOD framework to bypass safety alignment using OOD-ifying techniques
• Demonstrates high success rate in jailbreaking models like GPT-4

Innovation

• Uses out-of-distribution strategy for jailbreaking
• Applies visual and textual transformation techniques
• Leverages simple mixing-based techniques effectively