Scalable Evaluation of Online Moderation Strategies via Synthetic Simulations

2025-03-13

arXiv.org

AI Summary

Long-standing online discussion promotion strategies have lacked large-scale empirical evaluation, primarily due to prohibitively high human participation costs. This paper introduces SynDisco: the first fully LLM-driven synthetic discussion simulation framework, enabling concurrent simulation of hundreds of participants and circumventing human bottlenecks. Methodologically, it proposes a novel reinforcement learning–inspired LLM content regulation strategy; identifies that lightweight models better foster discussion diversity; and establishes an automated pipeline for instruction tuning, annotation, and evaluation. Key contributions include: (1) releasing VMD—the first open-source Virtual Moderation Dataset—comprising ten thousand multi-model–generated and expert-annotated samples; (2) demonstrating that our strategy significantly outperforms both human-crafted guidelines and off-the-shelf LLM regulation baselines; and (3) enabling efficient, scalable, and fully autonomous large-scale strategy evaluation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

Despite the ever-growing importance of online moderation, there has been no large-scale study evaluating the effectiveness of alternative moderation strategies. This is largely due to the lack of appropriate datasets, and the difficulty of getting human discussants, moderators, and evaluators involved in multiple experiments. In this paper, we propose a methodology for leveraging synthetic experiments performed exclusively by Large Language Models (LLMs) to initially bypass the need for human participation in experiments involving online moderation. We evaluate six LLM moderation configurations; two currently used real-life moderation strategies (guidelines issued for human moderators for online moderation and real-life facilitation), two baseline strategies (guidelines elicited for LLM alignment work, and LLM moderation with minimal prompting) a baseline with no moderator at all, as well as our own proposed strategy inspired by a Reinforcement Learning (RL) formulation of the problem. We find that our own moderation strategy significantly outperforms established moderation guidelines, as well as out-of-the-box LLM moderation. We also find that smaller LLMs, with less intensive instruction-tuning, can create more varied discussions than larger models. In order to run these experiments, we create and release an efficient, purpose-built, open-source Python framework, dubbed"SynDisco"to easily simulate hundreds of discussions using LLM user-agents and moderators. Additionally, we release the Virtual Moderation Dataset (VMD), a large dataset of LLM-generated and LLM-annotated discussions, generated by three families of open-source LLMs accompanied by an exploratory analysis of the dataset.

Problem

• Evaluating online facilitation strategies lacks large-scale cost-effective methods
• Testing if advanced facilitation strategies improve LLM-driven discussion quality
• Assessing performance differences between small and large LLMs in moderation

Innovation

• LLM-driven synthetic discussion simulation methodology
• Open-source framework SynDisco for implementation
• Virtual Moderation Dataset for public use