More Than Sum of Its Parts: Deciphering Intent Shifts in Multimodal Hate Speech Detection

2026-03-22

AI Summary

This work addresses the challenge of detecting hate speech that is implicitly conveyed through semantic interactions among neutral multimodal cues, particularly when intent diverges across modalities—a scenario where existing methods underperform. The study formally characterizes the intent misalignment problem in multimodal hate and introduces H-VLI, the first benchmark dataset focused on complex cross-modal semantic interactions rather than overtly offensive content. To tackle this, the authors propose ARCADE, a novel framework inspired by courtroom debate dynamics, which employs an asymmetric adversarial reasoning mechanism wherein “prosecution” and “defense” agents collaboratively dissect implicit hateful semantics. Experiments demonstrate that ARCADE substantially outperforms state-of-the-art approaches on H-VLI, especially in implicit cases, while maintaining competitive performance on established benchmarks.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Combating hate speech on social media is critical for securing cyberspace, yet relies heavily on the efficacy of automated detection systems. As content formats evolve, hate speech is transitioning from solely plain text to complex multimodal expressions, making implicit attacks harder to spot. Current systems, however, often falter on these subtle cases, as they struggle with multimodal content where the emergent meaning transcends the aggregation of individual modalities. To bridge this gap, we move beyond binary classification to characterize semantic intent shifts where modalities interact to construct implicit hate from benign cues or neutralize toxicity through semantic inversion. Guided by this fine-grained formulation, we curate the Hate via Vision-Language Interplay (H-VLI) benchmark where the true intent hinges on the intricate interplay of modalities rather than overt visual or textual slurs. To effectively decipher these complex cues, we further propose the Asymmetric Reasoning via Courtroom Agent DEbate (ARCADE) framework. By simulating a judicial process where agents actively argue for accusation and defense, ARCADE forces the model to scrutinize deep semantic cues before reaching a verdict. Extensive experiments demonstrate that ARCADE significantly outperforms state-of-the-art baselines on H-VLI, particularly for challenging implicit cases, while maintaining competitive performance on established benchmarks. Our code and data are available at: https://github.com/Sayur1n/H-VLI

Problem

• multimodal hate speech
• intent shift
• semantic interplay
• implicit hate
• hate speech detection

Innovation

• multimodal hate speech
• intent shift
• vision-language interplay
• asymmetric reasoning
• courtroom agent debate