Efficient Hate Speech Detection: A Three-Layer LoRA-Tuned BERTweet Framework

2025-11-08

AI Summary

To address the challenge of balancing real-time efficiency and detection performance for hate speech in resource-constrained environments, this paper proposes a three-tier collaborative framework: (i) rule-based lightweight pre-filtering; (ii) parameter-efficient fine-tuning of BERTweet via LoRA; and (iii) integration of a continual learning mechanism to adapt to evolving data distributions. The framework requires only 1.87M trainable parameters—just 1.37% of full fine-tuning—and completes training within two hours on a single T4 GPU, achieving a macro-F1 score of 0.85—94% of the performance attained by a 14B-parameter large language model. Its key contributions lie in the synergistic co-design across rule-based, model-level, and learning-level components; high accuracy preservation under extreme parameter efficiency; and an end-to-end lightweight pipeline tailored for edge deployment—significantly reducing computational overhead and deployment costs.

Technology Category

• Machine Learning: Hardware-aware ML
• Natural Language Processing: Safety and Robustness
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

This paper addresses the critical challenge of developing computationally efficient hate speech detection systems that maintain competitive performance while being practical for real-time deployment. We propose a novel three-layer framework that combines rule-based pre-filtering with a parameter-efficient LoRA-tuned BERTweet model and continuous learning capabilities. Our approach achieves 0.85 macro F1 score - representing 94% of the performance of state-of-the-art large language models like SafePhi (Phi-4 based) while using a base model that is 100x smaller (134M vs 14B parameters). Compared to traditional BERT-based approaches with similar computational requirements, our method demonstrates superior performance through strategic dataset unification and optimized fine-tuning. The system requires only 1.87M trainable parameters (1.37% of full fine-tuning) and trains in approximately 2 hours on a single T4 GPU, making robust hate speech detection accessible in resource-constrained environments while maintaining competitive accuracy for real-world deployment.

Problem

• Developing efficient hate speech detection for real-time deployment
• Maintaining competitive performance with smaller computational requirements
• Enabling robust detection in resource-constrained environments

Innovation

• Three-layer framework combining rule-based pre-filtering
• Parameter-efficient LoRA-tuned BERTweet model
• Continuous learning capabilities for hate speech detection