Hierarchical Multi-Stage BERT Fusion Framework with Dual Attention for Enhanced Cyberbullying Detection in Social Media

2025-03-01

AI Summary

To address the low accuracy and poor robustness in detecting cyberbullying content on social media—attributed to its semantically complex and dynamically evolving nature—this paper proposes a multi-stage BERT-based fusion framework. Methodologically, we design a hierarchical Transformer encoder to extract word-, sentence-, and document-level semantic embeddings, jointly incorporating sentiment and topic features; introduce self-attention and cross-modal cross-attention mechanisms to align heterogeneous information; and propose a novel hierarchical multi-stage classification head with a dynamic weighted loss strategy to enable fine-grained cyberbullying type identification. Extensive experiments on multiple public benchmarks demonstrate that our approach achieves significantly higher F1-scores than state-of-the-art models, with consistent improvements in precision, recall, and cross-domain generalization. The framework exhibits strong practical viability for real-world deployment.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Detecting and classifying cyberbullying in social media is hard because of the complex nature of online language and the changing nature of content. This study presents a multi-stage BERT fusion framework. It uses hierarchical embeddings, dual attention mechanisms, and extra features to improve detection of cyberbullying content. The framework combines BERT embeddings with features like sentiment and topic information. It uses self-attention and cross-attention to align features and has a hierarchical classification head for multi-category classification. A dynamic loss balancing strategy helps optimize learning and improves accuracy, precision, recall, and F1-score. These results show the model's strong performance and potential for broader use in analyzing social media content.

Problem

• Detects cyberbullying in complex social media language.
• Uses BERT fusion with hierarchical embeddings and dual attention.
• Improves accuracy with dynamic loss balancing strategy.

Innovation

• Multi-stage BERT fusion with hierarchical embeddings
• Dual attention mechanisms for feature alignment
• Dynamic loss balancing for optimized learning