Safe Alone, Unsafe Together: Safeguarding Against Implicit Toxicity When Benign Images Combine

2026-07-01

AI Summary

This study addresses the challenge that existing content moderation systems struggle to detect safety risks arising from multi-image combinations where individual images appear harmless but jointly convey harmful implicit semantics—a problem formalized here as Multi-Image Implicit Toxicity (MIIT). The work introduces a method for automatically constructing a multi-image safety benchmark, MIIT-dataset, and proposes MiShield-8B, a multimodal moderation model equipped with relational entity reasoning capabilities. By incorporating a progressive reasoning supervision mechanism, MiShield-8B simultaneously enhances interpretability and detection performance. Experimental results demonstrate that MiShield-8B outperforms leading commercial content moderation services and even larger-scale models on the MIIT detection task, confirming the effectiveness and practical utility of the proposed approach.

Abstract

Multi-image content has become an increasingly prevalent form of visual communication in social media, giving rise to a new safety issue, multi-image implicit toxicity (MIIT), where each image appears benign in isolation, but harmful semantics emerge when the images are interpreted jointly. MIIT is particularly challenging for existing commercial moderation APIs and models due to the lack of explicit risky cues in each image. This paper aims to study how to identify MIIT. We first provide a formal definition of MIIT and analyze three key challenges for its detection. To alleviate the scarcity of data in this area, we construct MIIT-dataset, an image-only multi-image safety dataset covering seven representative risk categories through an automatic generation pipeline. Finally, we train MiShield with progressively distilled reasoning supervision, enabling it to produce safety judgments accompanied by explicit analyses of the correlated entities that result in the hazards. Experiments show that MiShield-8B models outperform representative moderation services and even larger-scale models, revealing its effectiveness and practical value for this widely used visual format. Warning: This paper contains potentially sensitive content.

Problem

• multi-image implicit toxicity
• content moderation
• visual safety
• harmful semantics
• social media

Innovation

• multi-image implicit toxicity
• MIIT-dataset
• reasoning supervision
• content moderation
• visual safety