CLARA: Clip-Level Multimodal Alignment with VLM-Derived Rationales for Hateful Video Detection

2026-08-16

AI Summary

This study addresses the challenge of detecting implicit, temporally dependent multimodal signals in hateful videos, where conventional video-level representations often fail. We propose CLARA, a segment-level multimodal framework that employs a Mixture-of-Experts encoder and a gated Transformer for adaptive alignment. By integrating local-global contrastive learning with Vision-Language Model semantic guidance, CLARA effectively enhances the modeling of fine-grained temporal hate signals. Extensive experiments demonstrate that CLARA outperforms state-of-the-art methods across three benchmark datasets. Furthermore, ablation studies validate the efficacy of each core component, confirming significant improvements in detection performance. This work provides a robust solution for capturing subtle multimodal cues in complex video content, advancing the field of automated hate speech detection.

Abstract

Hateful video detection has become increasingly important with the rapid growth of video-centric social media platforms, given the serious risks that hate speech poses to both individual well-being and social cohesion. Compared with text or static multimodal content, hateful video detection remains underexplored and significantly more challenging, as hateful meaning often arises from complex interactions among multimodal cues, including speech, audio, and visual content. Moreover, such signals are often brief, implicit, and temporally dependent, making them difficult to capture using conventional video-level representations. In this work, we propose CLARA, a clip-level multimodal framework for hateful video detection. Instead of treating a video as a single instance, CLARA models it as a sequence of fine-grained clips, enabling more precise capture of temporally localized hateful signals. We introduce a Mixture-of-Experts clip encoder for adaptive multimodal alignment, a local-global segment contrastive objective to jointly model short-term cues and long-range temporal dependencies, and VLM-derived rationales integrated via a gated Transformer to provide high-level semantic guidance. Extensive experiments on three hateful video datasets demonstrate that CLARA consistently outperforms state-of-the-art methods. Further ablation studies and parameter analyses validate the effectiveness of each component.

Problem

• Hateful Video Detection
• Multimodal Alignment
• Temporal Dependency
• Clip-Level Representation

Innovation

• Clip-Level Multimodal Alignment
• Mixture-of-Experts
• Local-Global Segment Contrastive
• VLM-Derived Rationales
• Hateful Video Detection