Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directions

2025-09-29

AI Summary

Toxic behaviors in online content and AI systems—particularly large language models (LLMs)—endanger individual and societal well-being; their multimodal manifestations (text, image, video) and strong context dependence render existing taxonomies fragmented and reactive, impeding proactive governance. This study addresses this gap through a systematic literature review and multi-perspective analysis to develop the first comprehensive toxicity taxonomy integrating contextual, socio-technical, and modality-specific dimensions, thereby transcending traditional reactive paradigms. Methodologically, we propose an explainable detection framework tailored for LLMs, adaptive mitigation strategies, and a standardized evaluation protocol. We further synthesize major English-text toxicity datasets, model approaches, and critical research gaps. The contributions include: (1) a foundational theoretical taxonomy; (2) novel technical tools for interpretable detection and context-aware mitigation; and (3) actionable guidelines for scalable, evidence-based toxicity governance.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

The evolution of digital communication systems and the designs of online platforms have inadvertently facilitated the subconscious propagation of toxic behavior. Giving rise to reactive responses to toxic behavior. Toxicity in online content and Artificial Intelligence Systems has become a serious challenge to individual and collective well-being around the world. It is more detrimental to society than we realize. Toxicity, expressed in language, image, and video, can be interpreted in various ways depending on the context of usage. Therefore, a comprehensive taxonomy is crucial to detect and mitigate toxicity in online content, Artificial Intelligence systems, and/or Large Language Models in a proactive manner. A comprehensive understanding of toxicity is likely to facilitate the design of practical solutions for toxicity detection and mitigation. The classification in published literature has focused on only a limited number of aspects of this very complex issue, with a pattern of reactive strategies in response to toxicity. This survey attempts to generate a comprehensive taxonomy of toxicity from various perspectives. It presents a holistic approach to explain the toxicity by understanding the context and environment that society is facing in the Artificial Intelligence era. This survey summarizes the toxicity-related datasets and research on toxicity detection and mitigation for Large Language Models, social media platforms, and other online platforms, detailing their attributes in textual mode, focused on the English language. Finally, we suggest the research gaps in toxicity mitigation based on datasets, mitigation strategies, Large Language Models, adaptability, explainability, and evaluation.

Problem

• Developing a comprehensive taxonomy to detect and mitigate online toxicity
• Addressing reactive strategies and research gaps in toxicity mitigation
• Analyzing toxicity in AI systems, social media, and digital platforms

Innovation

• Develops comprehensive toxicity taxonomy from multiple perspectives
• Proposes holistic approach analyzing societal context and environment
• Summarizes datasets and detection methods for online platforms