Collective moderation of hate, toxicity, and extremity in online discussions

📅 2023-03-01
🏛️ PNAS Nexus
📈 Citations: 5
Influential: 2
📄 PDF

career value

182K/year
🤖 AI Summary
This study investigates the efficacy of citizen-initiated counter-speech in mitigating online hate, toxicity, and extremist discourse. Drawing on over 130,000 German-language Twitter conversations (2015–2019), it integrates human annotation, BERT-based sentiment and argumentation strategy classification, propensity score matching, and time-series analysis to conduct the first multi-level empirical evaluation—spanning micro-level (individual responses), meso-level (dialogue evolution), and macro-level (topic diffusion) dynamics. Key findings: (1) Direct, non-insulting articulation of opposing viewpoints demonstrates the most robust and generalizable deterrent effect; (2) Contextualized irony proves significantly effective against organized extremist groups—a result extending beyond prior small-sample studies. The work advances platform-enabled collaborative governance by identifying scalable, low-barrier collective moderation strategies grounded in grassroots counter-speech practices.
📝 Abstract
In the digital age, hate speech poses a threat to the functioning of social media platforms as spaces for public discourse. Top-down approaches to moderate hate speech encounter difficulties due to conflicts with freedom of expression and issues of scalability. Counter speech, a form of collective moderation by citizens, has emerged as a potential remedy. Here, we aim to investigate which counter speech strategies are most effective in reducing the prevalence of hate, toxicity, and extremity on online platforms. We analyze more than 130,000 discussions on German Twitter starting at the peak of the migrant crisis in 2015 and extending over four years. We use human annotation and machine learning classifiers to identify argumentation strategies, ingroup and outgroup references, emotional tone, and different measures of discourse quality. Using matching and time-series analyses we discern the effectiveness of naturally observed counter speech strategies on the micro-level (individual tweet pairs), meso-level (entire discussions) and macro-level (over days). We find that expressing straightforward opinions, even if not factual but devoid of insults, results in the least subsequent hate, toxicity, and extremity over all levels of analyses. This strategy complements currently recommended counter speech strategies and is easy for citizens to engage in. Sarcasm can also be effective in improving discourse quality, especially in the presence of organized extreme groups. Going beyond one-shot analyses on smaller samples prevalent in most prior studies, our findings have implications for the successful management of public online spaces through collective civic moderation.
Problem

Research questions and friction points this paper is trying to address.

Investigating effective counter speech strategies against online hate and toxicity
Analyzing collective moderation impact across micro, meso, and macro levels
Identifying practical approaches for citizens to improve online discourse quality
Innovation

Methods, ideas, or system contributions that make the work stand out.

Using human annotation and machine learning classifiers
Applying matching and time-series analyses across multiple levels
Identifying effective counter speech strategies through large-scale analysis
🔎 Similar Papers
No similar papers found.
J
J. Lasser
IDea_Lab, University of Graz, Graz, Austria.
A
Alina Herderich
Department of Computer Science and Biomedical Engineering, Graz University of Technology, Graz, Austria.
Joshua Garland
Joshua Garland
Arizona State University
Complex SystemsNatural Language ProcessingNonlinear Time Series Analysis
S
S. Aroyehun
Department of Politics and Public Administration, University of Konstanz, Konstanz, Germany.
D
D. Garcia
Complexity Science Hub Vienna, Vienna, Austria.
M
M. Galesic
Vermont Complex Systems Center, University of Vermont, Burlington, VT, United States.