FigSIM: A Dataset for Fine-grained Suicide Severity and Figurative Language in Suicide Memes

2026-06-01

AI Summary

Current content moderation systems struggle to detect high-risk suicide-related memes containing metaphorical expressions, primarily due to a lack of fine-grained annotated data. This work introduces FigSIM, the first multidimensional annotated dataset specifically designed for suicide-related memes, featuring labels across three dimensions: suicide severity, rhetorical devices, and related thematic content. The authors conduct a systematic benchmark evaluation of 16 unimodal and multimodal models on this dataset. Experimental results reveal that existing models consistently underestimate the risk of highly severe, metaphor-laden content, thereby highlighting FigSIM’s critical role in uncovering blind spots in current modeling approaches and advancing more precise and nuanced content moderation strategies.

Abstract

Suicide memes are memes used to express suicide-related thoughts or comment on suicide-related issues. Suicide memes are increasingly common on social media, yet remain poorly understood and potentially harmful. There is an urgent need to better understand their characteristics and to develop appropriate content moderation strategies that limits users' exposure to potentially harmful content. Currently, the absence of annotated datasets of suicide memes remains a key barrier to developing and evaluating automated moderation approaches. In this paper, we introduce FigSIM, the first dataset designed for fine-grained analysis of suicide memes. The dataset consists of 1049 memes, each annotated for (1) fine-grained suicide severity levels, (2) figurative phenomena (e.g., metaphors), and (3) suicide-related content (e.g., suicide method depiction). We benchmark 16 unimodal and multimodal models across three tasks: figurative language, suicide severity, and suicide-related content detection. Overall, FigSIM demonstrates that suicide memes pose unique challenges for both modeling and content moderation. Analysis revealed biases, such as underprediction of higher suicide severity levels, especially for figurative memes. The dataset (including splits used for analyses) is publicly available. Content Warning: This paper contains suicide-related content that may be triggering.

Problem

• suicide memes
• fine-grained severity
• figurative language
• content moderation
• annotated dataset

Innovation

• suicide memes
• fine-grained severity
• figurative language
• multimodal dataset
• content moderation