They Said Memes Were Harmless-We Found the Ones That Hurt: Decoding Jokes, Symbols, and Cultural References

2026-02-03

AI Summary

This study addresses three key challenges in detecting abusive meme content: the lack of cultural context, the blurred boundary between satire and abuse, and limited model interpretability. To tackle these issues, the authors propose CROSS-ALIGN+, a three-stage framework that integrates structured knowledge from ConceptNet, Wikidata, and Hatebase to enhance cultural understanding, employs parameter-efficient LoRA adapters to precisely distinguish satire from abusive content, and introduces a cascaded explanation mechanism to improve decision transparency. Extensive experiments across five benchmark datasets and eight large vision-language models demonstrate that the proposed method achieves up to a 17% relative improvement in F1 score over state-of-the-art approaches while providing traceable and interpretable justifications for its predictions.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Meme-based social abuse detection is challenging because harmful intent often relies on implicit cultural symbolism and subtle cross-modal incongruence. Prior approaches, from fusion-based methods to in-context learning with Large Vision-Language Models (LVLMs), have made progress but remain limited by three factors: i) cultural blindness (missing symbolic context), ii) boundary ambiguity (satire vs. abuse confusion), and iii) lack of interpretability (opaque model reasoning). We introduce CROSS-ALIGN+, a three-stage framework that systematically addresses these limitations: (1) Stage I mitigates cultural blindness by enriching multimodal representations with structured knowledge from ConceptNet, Wikidata, and Hatebase; (2) Stage II reduces boundary ambiguity through parameter-efficient LoRA adapters that sharpen decision boundaries; and (3) Stage III enhances interpretability by generating cascaded explanations. Extensive experiments on five benchmarks and eight LVLMs demonstrate that CROSS-ALIGN+ consistently outperforms state-of-the-art methods, achieving up to 17% relative F1 improvement while providing interpretable justifications for each decision.

Problem

• meme-based abuse
• cultural symbolism
• cross-modal incongruence
• harmful intent detection
• social media moderation

Innovation

• meme-based abuse detection
• cultural context
• LoRA adapters
• interpretable AI
• multimodal reasoning