All Changes May Have Invariant Principles: Improving Ever-Shifting Harmful Meme Detection via Design Concept Reproduction

2026-01-08

arXiv.org

AI Summary

This work addresses the challenge of detecting harmful internet memes, which exhibit diverse types and evolve over time, rendering existing detection methods ineffective. The authors propose the first detection framework grounded in design concept replication, leveraging invariant principles underlying malicious creators’ strategies to construct a Design Concept Graph (DCG). By integrating attack tree methodology to model and prune multimodal content, the framework guides a Multimodal Large Language Model (MLLM) toward generalized reasoning. The approach achieves a detection accuracy of 81.1% and demonstrates robust performance under type migration and temporal evolution scenarios. Human evaluations further indicate that the analysis efficiency per meme is improved to 15–30 seconds.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Computer Vision: Multi-modal Vision

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Harmful memes are ever-shifting in the Internet communities, which are difficult to analyze due to their type-shifting and temporal-evolving nature. Although these memes are shifting, we find that different memes may share invariant principles, i.e., the underlying design concept of malicious users, which can help us analyze why these memes are harmful. In this paper, we propose RepMD, an ever-shifting harmful meme detection method based on the design concept reproduction. We first refer to the attack tree to define the Design Concept Graph (DCG), which describes steps that people may take to design a harmful meme. Then, we derive the DCG from historical memes with design step reproduction and graph pruning. Finally, we use DCG to guide the Multimodal Large Language Model (MLLM) to detect harmful memes. The evaluation results show that RepMD achieves the highest accuracy with 81.1% and has slight accuracy decreases when generalized to type-shifting and temporal-evolving memes. Human evaluation shows that RepMD can improve the efficiency of human discovery on harmful memes, with 15$\sim$30 seconds per meme.

Problem

• harmful memes
• ever-shifting
• type-shifting
• temporal-evolving
• meme detection

Innovation

• Design Concept Graph
• Harmful Meme Detection
• Multimodal Large Language Model
• Invariant Principles
• Attack Tree