Untangling Hate Speech Definitions: A Semantic Componential Analysis Across Cultures and Domains

2024-11-11

North American Chapter of the Association for Computational Linguistics

AI Summary

This study addresses the lack of cultural adaptability and definitional diversity of hate speech across cross-cultural and cross-domain contexts. We construct the first hate speech definition dataset covering 100+ cultures and five domains (493 definitions) and propose Semantic Component Analysis (SCA), the first framework enabling multi-cultural, multi-source semantic disentanglement of definitions. Through zero-shot prompting experiments with LLaMA, Phi, and Qwen, we empirically demonstrate that definitional complexity significantly degrades detection performance, and while legal and platform definitions are frequently borrowed across domains, they exhibit poor cultural adaptation. Our core contributions are threefold: (1) establishing the first cross-cultural, cross-domain hate speech definition benchmark; (2) introducing an interpretable, semantics-driven definition decomposition method; and (3) providing empirical evidence that definition selection critically impacts AI ethics evaluation—thereby laying theoretical and practical foundations for culturally sensitive content governance.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Application Domains: Humanities & Computational Social Science
• Machine Learning: Ethics, Bias, and Fairness

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Hate speech relies heavily on cultural influences, leading to varying individual interpretations. For that reason, we propose a Semantic Componential Analysis (SCA) framework for a cross-cultural and cross-domain analysis of hate speech definitions. We create the first dataset of hate speech definitions encompassing 493 definitions from more than 100 cultures, drawn from five key domains: online dictionaries, academic research, Wikipedia, legal texts, and online platforms. By decomposing these definitions into semantic components, our analysis reveals significant variation across definitions, yet many domains borrow definitions from one another without taking into account the target culture. We conduct zero-shot model experiments using our proposed dataset, employing three popular open-sourced LLMs to understand the impact of different definitions on hate speech detection. Our findings indicate that LLMs are sensitive to definitions: responses for hate speech detection change according to the complexity of definitions used in the prompt.

Problem

• Analyzing cultural variations in hate speech definitions
• Developing a cross-cultural hate speech dataset
• Assessing LLM sensitivity to definition differences

Innovation

• Semantic Componential Analysis for hate speech definitions
• First dataset with 493 definitions from 100 cultures
• Zero-shot experiments with LLMs on definition sensitivity