KID: Knowledge-Injected Dual-Head Learning for Knowledge-Grounded Harmful Meme Detection

2026-01-29

AI Summary

This work addresses the challenge of detecting implicitly harmful content in internet memes, which often relies on metaphors and sociocultural context that current methods struggle to interpret. To this end, we propose KID, a novel framework that explicitly injects external knowledge into meme understanding by constructing a knowledge-guided, structured reasoning chain that links visual evidence, contextual knowledge, and classification labels. KID employs a dual-headed neural architecture optimized via label-constrained distillation to jointly refine semantic generation and classification objectives, while supporting multilingual cross-modal comprehension. Evaluated across five datasets—including English, Chinese, and Bengali—KID achieves state-of-the-art performance on both binary and multi-label tasks, outperforming previous best methods by 2.1% to 19.7% on key metrics.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Internet memes have become pervasive carriers of digital culture on social platforms. However, their heavy reliance on metaphors and sociocultural context also makes them subtle vehicles for harmful content, posing significant challenges for automated content moderation. Existing approaches primarily focus on intra-modal and inter-modal signal analysis, while the understanding of implicit toxicity often depends on background knowledge that is not explicitly present in the meme itself. To address this challenge, we propose KID, a Knowledge-Injected Dual-Head Learning framework for knowledge-grounded harmful meme detection. KID adopts a label-constrained distillation paradigm to decompose complex meme understanding into structured reasoning chains that explicitly link visual evidence, background knowledge, and classification labels. These chains guide the learning process by grounding external knowledge in meme-specific contexts. In addition, KID employs a dual-head architecture that jointly optimizes semantic generation and classification objectives, enabling aligned linguistic reasoning while maintaining stable decision boundaries. Extensive experiments on five multilingual datasets spanning English, Chinese, and low-resource Bengali demonstrate that KID achieves SOTA performance on both binary and multi-label harmful meme detection tasks, improving over previous best methods by 2.1%--19.7% across primary evaluation metrics. Ablation studies further confirm the effectiveness of knowledge injection and dual-head joint learning, highlighting their complementary contributions to robust and generalizable meme understanding. The code and data are available at https://github.com/PotatoDog1669/KID.

Problem

• harmful meme detection
• background knowledge
• implicit toxicity
• content moderation
• multimodal understanding

Innovation

• knowledge injection
• dual-head learning
• structured reasoning chains
• multilingual harmful meme detection
• label-constrained distillation