CounterQuill: Investigating the Potential of Human-AI Collaboration in Online Counterspeech Writing

2024-10-03

arXiv.org

AI Summary

Online hate speech proliferates, yet existing countermeasures are hindered by users’ fear of retaliation and high technical barriers. This paper introduces CounterQuill, an AI-mediated collaborative writing system that establishes the first human-AI co-writing framework specifically designed for countering hate speech. CounterQuill employs a three-phase workflow—learning, brainstorming, and co-writing—integrating structured prompt engineering with cognitively scaffolded, stepwise guidance. This design preserves user agency and emotional resonance while addressing the limitations of general-purpose large language models in value-sensitive tasks, particularly their insufficient guidance and collaborative adaptability. A user study demonstrates that, compared to ChatGPT, CounterQuill significantly enhances users’ sense of ownership over generated counter-speech and increases their willingness to publish it—thereby validating the effectiveness and feasibility of collaborative AI in value-laden domains.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Application Domains: Humanities & Computational Social Science
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Online hate speech has become increasingly prevalent on social media platforms, causing harm to individuals and society. While efforts have been made to combat this issue through content moderation, the potential of user-driven counterspeech as an alternative solution remains underexplored. Existing counterspeech methods often face challenges such as fear of retaliation and skill-related barriers. To address these challenges, we introduce CounterQuill, an AI-mediated system that assists users in composing effective and empathetic counterspeech. CounterQuill provides a three-step process: (1) a learning session to help users understand hate speech and counterspeech; (2) a brainstorming session that guides users in identifying key elements of hate speech and exploring counterspeech strategies; and (3) a co-writing session that enables users to draft and refine their counterspeech with CounterQuill. We conducted a within-subjects user study with 20 participants to evaluate CounterQuill in comparison to ChatGPT. Results show that CounterQuill's guidance and collaborative writing process provided users a stronger sense of ownership over their co-authored counterspeech. Users perceived CounterQuill as a writing partner and thus were more willing to post the co-written counterspeech online compared to the one written with ChatGPT.

Problem

• Addressing online hate speech through user-driven counterspeech
• Overcoming fear and skill barriers in counterspeech writing
• Enhancing counterspeech effectiveness via AI-human collaboration

Innovation

• AI-mediated system for counterspeech writing
• Three-step process: learning, brainstorming, co-writing
• Enhances user ownership and willingness to post