Adapting Reinforcement Learning with Chain-of-Thought Supervision for Explainable Detection of Hateful and Propagandistic Memes

2026-06-13

AI Summary

This work addresses the challenge of hateful meme detection exacerbated by the modality gap between image and text, proposing a reinforcement learning–based post-training approach for multimodal large language models. It introduces Group Relative Policy Optimization (GRPO) combined with chain-of-thought supervision, marking the first application of chain-of-thought distillation and GRPO to meme moderation. The method features a joint optimization objective that integrates classification accuracy with explanation quality, augmented by thought-length regularization. Furthermore, it explores a self-supervised GRPO strategy leveraging unlabeled data. Evaluated on the Hateful Memes and ArMeme benchmarks, the approach achieves 82.0% FHM accuracy—an improvement of 2.1 percentage points—and a macro F1 score of 0.612, gaining 7.6 points over prior methods, while simultaneously generating high-quality natural language explanations.

Abstract

Hateful and propagandistic memes exploit the interplay between images and text to convey harmful intent that neither modality reveals alone. Although thinking-based multimodal large language models (MLLMs) have advanced vision-language understanding, their application to meme content moderation remains underexplored. We propose a reinforcement learning-based post-training method that improves classification performance and reference-based explanation quality in thinking-based MLLMs via task-specific rewards and Group Relative Policy Optimization (GRPO). Concretely, we (i) conduct a systematic empirical study of off-the-shelf MLLMs for hateful and propagandistic meme understanding across English and Arabic benchmarks, (ii) extend existing meme datasets with weakly supervised chain-of-thought (CoT) rationales via distillation and multi-LLM fine-grained propaganda annotations, (iii) introduce a GRPO-based objective with thinking-length regularization that jointly optimizes classification accuracy and explanation quality, and (iv) investigate self-supervised GRPO on unlabeled memes using consensus-based pseudo-labels. Experiments on the Hateful Memes and ArMeme benchmarks show that our approach improves over previously reported results on FHM accuracy (up to +2.1%, from 79.9% to 82.0%) and on ArMeme macro-F1 (up to +7.6 points, from 0.536 to 0.612 with explanations; +6.1 compared to the original ArMeme benchmark), while also generating natural-language explanations. On ArMeme, sequence-classification baselines remain stronger in terms of raw accuracy, whereas our approach provides more balanced per-class performance along with explanations. We publicly release our code, data extensions, and evaluation resources.

Problem

• hateful memes
• propagandistic memes
• multimodal understanding
• explainable detection
• content moderation

Innovation

• Reinforcement Learning
• Chain-of-Thought
• Multimodal Large Language Models
• Group Relative Policy Optimization
• Explainable AI