IYKYK: Using language models to decode extremist cryptolects

2025-06-05

AI Summary

Extremist groups employ cryptolects to evade content moderation, yet current large language models (LLMs) exhibit significant limitations in detecting and semantically parsing such obfuscated language. Method: This work first systematically uncovers the failure mechanisms of general-purpose LLMs in understanding extremist cryptolects; proposes a domain-adaptive fine-tuning framework tailored to extremist contexts and an expert-guided structured prompting strategy; and constructs—then open-sources—the first manually validated, large-scale extremist platform corpus (19.4M posts) alongside a curated cryptolect lexicon. Contribution/Results: Our approach achieves substantial accuracy improvements across six benchmark tasks—including cryptolect detection, semantic decoding, and contextual interpretation—demonstrating robust performance gains over baseline LLMs. The released resources and reproducible methodology provide a verifiable, scalable foundation for automated extremist content moderation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Computer Vision: Large Vision Models

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Extremist groups develop complex in-group language, also referred to as cryptolects, to exclude or mislead outsiders. We investigate the ability of current language technologies to detect and interpret the cryptolects of two online extremist platforms. Evaluating eight models across six tasks, our results indicate that general purpose LLMs cannot consistently detect or decode extremist language. However, performance can be significantly improved by domain adaptation and specialised prompting techniques. These results provide important insights to inform the development and deployment of automated moderation technologies. We further develop and release novel labelled and unlabelled datasets, including 19.4M posts from extremist platforms and lexicons validated by human experts.

Problem

• Detecting extremist cryptolects using language models
• Evaluating general LLMs on extremist language tasks
• Improving performance via domain adaptation and prompting

Innovation

• Domain adaptation improves extremist language detection
• Specialized prompting enhances cryptolect decoding
• Novel datasets enable automated moderation development