See, Explain, and Intervene: A Few-Shot Multimodal Agent Framework for Hateful Meme Moderation

2026-01-08

arXiv.org

AI Summary

This work addresses the challenges of detecting, explaining, and preemptively mitigating hateful memes in data-scarce scenarios by proposing the first unified few-shot multimodal agent framework. The framework integrates task-specific generative multimodal agents with few-shot-adapted large multimodal models to simultaneously achieve hate content detection, semantic interpretation, and pre-publication intervention under extremely limited labeled data. By unifying these three critical tasks within a single generalizable and low-resource-friendly architecture, this study significantly enhances the efficiency, interpretability, and practical deployability of content moderation systems.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

In this work, we examine hateful memes from three complementary angles - how to detect them, how to explain their content and how to intervene them prior to being posted - by applying a range of strategies built on top of generative AI models. To the best of our knowledge, explanation and intervention have typically been studied separately from detection, which does not reflect real-world conditions. Further, since curating large annotated datasets for meme moderation is prohibitively expensive, we propose a novel framework that leverages task-specific generative multimodal agents and the few-shot adaptability of large multimodal models to cater to different types of memes. We believe this is the first work focused on generalizable hateful meme moderation under limited data conditions, and has strong potential for deployment in real-world production scenarios. Warning: Contains potentially toxic contents.

Problem

• hateful meme moderation
• few-shot learning
• multimodal AI
• content intervention
• explainable AI

Innovation

• few-shot learning
• multimodal agent
• hateful meme moderation
• generative AI
• content intervention