Post Guidance for Online Communities

2024-11-25

arXiv.org

AI Summary

Addressing the governance challenge of simultaneously ensuring content quality, sustaining user engagement, and minimizing moderator workload in online communities, this paper proposes a *posting guidance* mechanism: real-time, rule-based, personalized interventions—such as prompts, blocks, or flags—triggered during the authoring phase to enable proactive, lightweight, and policy-compliant content moderation. Our key contribution is a novel, rule-driven, community-customizable real-time guidance paradigm tailored to the posting process, powered by a dynamic rule engine that jointly leverages content and account features, rigorously evaluated via A/B testing and multi-dimensional behavioral analysis. Empirical deployment across 33 Reddit subreddits demonstrates statistically significant improvements: increased successful post submission rates, a 27.4% reduction in manual review volume, enhanced comment quality (e.g., depth and civility), and no significant decline in users’ subsequent engagement.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Application Domains: Humanities & Computational Social Science
• Humans and AI: Game Design — Procedural Content Generation & Storytelling

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Responsible Web: Mitigating online harms, including harassment, spam, and fake reviews
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Effective content moderation in online communities is often a delicate balance between maintaining content quality and fostering user participation. In this paper, we introduce post guidance, a novel approach to community moderation that proactively guides users' contributions using rules that trigger interventions as users draft a post to be submitted. For instance, rules can surface messages to users, prevent post submissions, or flag posted content for review. This uniquely community-specific, proactive, and user-centric approach can increase adherence to rules without imposing additional burdens on moderators. We evaluate a version of Post Guidance implemented on Reddit, which enables the creation of rules based on both post content and account characteristics, via a large randomized experiment, capturing activity from 97,616 posters in 33 subreddits over 63 days. We find that Post Guidance (1) increased the number of ``successful posts'' (posts not removed after 72 hours), (2) decreased moderators' workload in terms of manually-reviewed reports, (3) increased contribution quality, as measured by community engagement, and (4) had no impact on posters' own subsequent activity, within communities adopting the feature. Post Guidance on Reddit was similarly effective for community veterans and newcomers, with greater benefits in communities that used the feature more extensively. Our findings indicate that post guidance represents a transformative approach to content moderation, embodying a paradigm that can be easily adapted to other platforms to improve online communities across the Web.

Problem

• Content Quality
• User Engagement
• Moderator Overload

Innovation

• Posting Guidance
• Content Quality Improvement
• User Behavior Neutral Impact