LAMP: Learning Universal Adversarial Perturbations for Multi-Image Tasks via Pre-trained Models

2026-01-29

AI Summary

This work addresses the limitations of existing adversarial attack methods, which are primarily designed for single-image inputs under white-box settings and thus struggle to generalize to black-box scenarios involving multi-image multimodal large language models (MLLMs). To bridge this gap, we propose LAMP—the first black-box universal adversarial perturbation method tailored for multi-image MLLMs. LAMP introduces a cross-image contagion mechanism and a position-invariant index-attention suppression loss, enabling effective attacks by perturbing only a subset of input images. By leveraging attention constraints, our approach enhances both the transferability and imperceptibility of the generated perturbations. Extensive experiments demonstrate that LAMP significantly outperforms state-of-the-art methods across multiple vision-language tasks and mainstream MLLMs, achieving substantially higher attack success rates.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Multimodal Large Language Models (MLLMs) have achieved remarkable performance across vision-language tasks. Recent advancements allow these models to process multiple images as inputs. However, the vulnerabilities of multi-image MLLMs remain unexplored. Existing adversarial attacks focus on single-image settings and often assume a white-box threat model, which is impractical in many real-world scenarios. This paper introduces LAMP, a black-box method for learning Universal Adversarial Perturbations (UAPs) targeting multi-image MLLMs. LAMP applies an attention-based constraint that prevents the model from effectively aggregating information across images. LAMP also introduces a novel cross-image contagious constraint that forces perturbed tokens to influence clean tokens, spreading adversarial effects without requiring all inputs to be modified. Additionally, an index-attention suppression loss enables a robust position-invariant attack. Experimental results show that LAMP outperforms SOTA baselines and achieves the highest attack success rates across multiple vision-language tasks and models.

Problem

• multi-image MLLMs
• adversarial attacks
• black-box
• Universal Adversarial Perturbations
• vision-language tasks

Innovation

• Universal Adversarial Perturbations
• Multi-Image MLLMs
• Black-Box Attack
• Cross-Image Contagious Constraint
• Attention-Based Constraint