Are Open-Weight LLMs Ready for Social Media Moderation? A Comparative Study on Bluesky

2026-02-05

AI Summary

This study evaluates the practical efficacy of open-source large language models (LLMs) for content moderation on social media, with a focus on privacy-preserving and locally deployable scenarios. Leveraging real user posts from the Bluesky platform—annotated using both platform-provided labels and human raters—it presents the first systematic comparison of zero-shot performance between three open-source and four closed-source LLMs on harmful content detection. The results demonstrate that open-source models achieve sensitivity (81%–97%) and specificity (91%–100%) comparable to their closed-source counterparts, while operating efficiently on consumer-grade hardware. These findings substantiate the feasibility and deployment potential of open-source LLMs for both personalized and platform-scale content moderation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

As internet access expands, so does exposure to harmful content, increasing the need for effective moderation. Research has demonstrated that large language models (LLMs) can be effectively utilized for social media moderation tasks, including harmful content detection. While proprietary LLMs have been shown to zero-shot outperform traditional machine learning models, the out-of-the-box capability of open-weight LLMs remains an open question. Motivated by recent developments of reasoning LLMs, we evaluate seven state-of-the-art models: four proprietary and three open-weight. Testing with real-world posts on Bluesky, moderation decisions by Bluesky Moderation Service, and annotations by two authors, we find a considerable degree of overlap between the sensitivity (81%--97%) and specificity (91%--100%) of the open-weight LLMs and those (72%--98%, and 93%--99%) of the proprietary ones. Additionally, our analysis reveals that specificity exceeds sensitivity for rudeness detection, but the opposite holds for intolerance and threats. Lastly, we identify inter-rater agreement across human moderators and the LLMs, highlighting considerations for deploying LLMs in both platform-scale and personalized moderation contexts. These findings show open-weight LLMs can support privacy-preserving moderation on consumer-grade hardware and suggest new directions for designing moderation systems that balance community values with individual user preferences.

Problem

• open-weight LLMs
• social media moderation
• harmful content detection
• content moderation
• LLM evaluation

Innovation

• open-weight LLMs
• social media moderation
• harmful content detection
• zero-shot evaluation
• privacy-preserving moderation