Transfer Learning via Lexical Relatedness: A Sarcasm and Hate Speech Case Study

2025-08-22

AI Summary

Detecting implicit hate speech—such as ironic or allusive expressions—remains challenging due to its reliance on contextual and pragmatic cues rather than overt lexical markers. Method: This paper proposes a two-stage transfer learning framework that leverages sarcasm detection as a proxy task, capitalizing on shared semantic indirectness between sarcasm and implicit hate. We employ both CNN+LSTM and BERT+BiLSTM architectures, pretraining on Sarcasm on Reddit and the Implicit Hate Corpus, then fine-tuning on the ETHOS dataset (covering both explicit and implicit hate). Contribution/Results: Experimental results show that BERT+BiLSTM achieves substantial improvements on ETHOS: +9.7% recall, +7.8% AUC, and +6.0% F1-score overall; notably, precision on the implicit subset increases by 7.8%. This work is the first to systematically validate the efficacy of sarcasm-based pretraining for multi-level hate speech detection, revealing the transferable mechanism underlying semantic indirectness modeling and establishing a novel paradigm for detecting implicit harmful content.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Multiagent Systems: Adversarial Agents

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Detecting hate speech in non-direct forms, such as irony, sarcasm, and innuendos, remains a persistent challenge for social networks. Although sarcasm and hate speech are regarded as distinct expressions, our work explores whether integrating sarcasm as a pre-training step improves implicit hate speech detection and, by extension, explicit hate speech detection. Incorporating samples from ETHOS, Sarcasm on Reddit, and Implicit Hate Corpus, we devised two training strategies to compare the effectiveness of sarcasm pre-training on a CNN+LSTM and BERT+BiLSTM model. The first strategy is a single-step training approach, where a model trained only on sarcasm is then tested on hate speech. The second strategy uses sequential transfer learning to fine-tune models for sarcasm, implicit hate, and explicit hate. Our results show that sarcasm pre-training improved the BERT+BiLSTM's recall by 9.7%, AUC by 7.8%, and F1-score by 6% on ETHOS. On the Implicit Hate Corpus, precision increased by 7.8% when tested only on implicit samples. By incorporating sarcasm into the training process, we show that models can more effectively detect both implicit and explicit hate.

Problem

• Detecting non-direct hate speech like sarcasm
• Improving implicit hate speech detection via sarcasm pre-training
• Evaluating transfer learning effectiveness on hate speech models

Innovation

• Using sarcasm pre-training for hate detection
• Sequential transfer learning with BERT+BiLSTM model
• Lexical relatedness transfer between sarcasm and hate