Understanding Interpretation Difficulty in Harmful Online Communication: Insights from Cybercrime Communities

2026-07-08

AI Summary

This study addresses the challenge of interpreting harmful online content, which often employs slang, coded language, and community-specific expressions that obscure semantic meaning. Departing from conventional message-level classification, the work reframes harmful content analysis as an evidence integration problem. The authors construct reference interpretations through expert annotation and design context-controlled experiments to systematically evaluate the performance of both humans and large language models in deciphering criminal discussions on Discord. They propose the first interpretability difficulty taxonomy, revealing that local contextual cues are generally insufficient for accurate interpretation. The findings demonstrate that incorporating external knowledge and expanding conversational context significantly improves accuracy, with larger models consistently outperforming smaller ones.

Abstract

Harmful online communication often contains slang, coded terms, abbreviations, and community-specific expressions, which make messages difficult to interpret. This paper presents an exploratory study of interpretation difficulty in Discord chats related to cybercrime. We construct reference interpretations of purposefully selected difficult messages, which were reviewed by an expert. We then use them to evaluate human and large language model (LLM) interpretations under different context conditions. The results show that local context alone is often insufficient for humans, while external knowledge and extended conversational context substantially improve human interpretation. For LLMs, local context also improves interpretation, and the larger model performs better. We further conduct a qualitative error analysis and propose a preliminary classification of factors that make harmful chats difficult to interpret. These findings suggest that harmful-content analysis should treat interpretation as an evidence-integration problem, rather than as message-level classification alone.

Problem

• harmful online communication
• interpretation difficulty
• cybercrime communities
• coded language
• contextual understanding

Innovation

• interpretation difficulty
• harmful online communication
• large language models
• context integration
• cybercrime communities