Boosting Accuracy and Interpretability in Multilingual Hate Speech Detection Through Layer Freezing and Explainable AI

2026-01-06

arXiv.org

AI Summary

This study addresses the challenge of balancing model performance and interpretability in multilingual hate speech detection. The authors propose a novel approach that freezes the first eight layers of multilingual Transformer models—such as BERT-base-multilingual-cased, RoBERTa-base, and XLM-RoBERTa-base—and integrates the LIME interpretability framework. This work presents the first synergistic application of layer-freezing strategies and local explanation methods in the context of multilingual hate speech detection. Evaluated on English, Korean, Japanese, Chinese, and French datasets, the method significantly improves accuracy, precision, recall, and F1 scores while simultaneously providing word-level explanations for model predictions, thereby achieving both high efficiency and transparency.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Transparent, Interpretable, Explainable ML
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Sentiment analysis focuses on identifying the emotional polarity expressed in textual data, typically categorized as positive, negative, or neutral. Hate speech detection, on the other hand, aims to recognize content that incites violence, discrimination, or hostility toward individuals or groups based on attributes such as race, gender, sexual orientation, or religion. Both tasks play a critical role in online content moderation by enabling the detection and mitigation of harmful or offensive material, thereby contributing to safer digital environments. In this study, we examine the performance of three transformer-based models: BERT-base-multilingual-cased, RoBERTa-base, and XLM-RoBERTa-base with the first eight layers frozen, for multilingual sentiment analysis and hate speech detection. The evaluation is conducted across five languages: English, Korean, Japanese, Chinese, and French. The models are compared using standard performance metrics, including accuracy, precision, recall, and F1-score. To enhance model interpretability and provide deeper insight into prediction behavior, we integrate the Local Interpretable Model-agnostic Explanations (LIME) framework, which highlights the contribution of individual words to the models decisions. By combining state-of-the-art transformer architectures with explainability techniques, this work aims to improve both the effectiveness and transparency of multilingual sentiment analysis and hate speech detection systems.

Problem

• hate speech detection
• multilingual
• interpretability
• accuracy
• content moderation

Innovation

• layer freezing
• explainable AI
• multilingual hate speech detection
• transformer models
• LIME