Conversation-Based Multimodal Abuse Detection Through Text and Graph Embeddings

2025-03-17

AI Summary

To address the limitation of existing methods in abusive behavior detection on social platforms—namely, their neglect of the dynamic conversational structure—this paper proposes a multimodal embedding framework that jointly models textual semantics and message interaction graph topology. We introduce, for the first time, a full-graph embedding method supporting directed, weighted, signed edges and node attributes, and design three text–graph fusion strategies. Our framework integrates five textual embeddings (e.g., BERT) with thirteen graph embeddings—including novel self-developed variants—and incorporates interpretability analysis to identify discriminative features. Experimental results demonstrate that the best unimodal performance achieves F1-scores of 81.02 (text) and 80.61 (graph), while the proposed multimodal fusion attains 87.06—marking a substantial improvement over state-of-the-art approaches.

Technology Category

• Machine Learning: Multimodal Learning
• Natural Language Processing: Safety and Robustness
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Abusive behavior is common on online social networks, and forces the hosts of such platforms to find new solutions to address this problem. Various methods have been proposed to automate this task in the past decade. Most of them rely on the exchanged content, but ignore the structure and dynamics of the conversation, which could provide some relevant information. In this article, we propose to use representation learning methods to automatically produce embeddings of this textual content and of the conversational graphs depicting message exchanges. While the latter could be enhanced by including additional information on top of the raw conversational structure, no method currently exists to learn wholegraph representations using simultaneously edge directions, weights, signs, and vertex attributes. We propose two such methods to fill this gap in the literature. We experiment with 5 textual and 13 graph embedding methods, and apply them to a dataset of online messages annotated for abuse detection. Our best results achieve an F -measure of 81.02 using text alone and 80.61 using graphs alone. We also combine both modalities of information (text and graphs) through three fusion strategies, and show that this strongly improves abuse detection performance, increasing the F -measure to 87.06. Finally, we identify which specific engineered features are captured by the embedding methods under consideration. These features have clear interpretations and help explain what information the representation learning methods deem discriminative.

Problem

• Detect abusive behavior in online social networks.
• Combine text and conversational graph embeddings for abuse detection.
• Improve abuse detection performance using multimodal fusion strategies.

Innovation

• Uses text and graph embeddings for abuse detection.
• Develops methods for whole-graph representation learning.
• Combines text and graph data to enhance detection accuracy.