ControBench: An Interaction-Aware Benchmark for Controversial Discourse Analysis on Social Networks

2026-05-01

AI Summary

Existing datasets struggle to simultaneously capture the semantic content, interaction structure, and user ideology inherent in contentious social media discussions. To address this gap, this work introduces a novel benchmark dataset constructed from Reddit discussions on three polarizing topics—Trump, abortion, and religion—that uniquely integrates a heterogeneous user–post graph, interaction edges preserving local argumentative context, and scalable ideological labels derived from users’ self-declared affiliations. This design authentically reflects the low homophily characteristic of cross-ideological dialogue. Comprehensive evaluations using graph neural networks, pretrained language models, and large language models reveal topic- and model-dependent performance variations, with particularly pronounced challenges when ideological boundaries are ambiguous, thereby demonstrating the benchmark’s real-world relevance and technical complexity.

Abstract

Understanding how people argue across ideological divides online is important for studying political polarization, misinformation, and content moderation. Existing datasets capture only part of this problem: some preserve text but ignore interaction structure, some model structure without rich semantics, and others represent conversations without stable user-level ideological identity. We introduce ControBench, a benchmark for controversial discourse analysis that combines heterogeneous social interaction graphs with rich textual semantics. Built from Reddit discussions on three topics, Trump, abortion, and religion, ControBench contains 7,370 users, 1,783 posts, and 26,525 interactions. The graph contains user and post nodes connected by semantically enriched edges; in particular, user-comment-user edges encode both a reply and the parent comment that it responds to, preserving local argumentative context. User labels are derived from self-declared Reddit flairs, providing a scalable proxy for ideological identity without manual annotation. The resulting datasets exhibit low or negative adjusted homophily (Trump: -0.77, Abortion: 0.06, Religion: 0.04), reflecting the cross-cutting structure of real-world debate. We evaluate graph neural networks, pretrained language models, and large language models on ControBench and observe distinct performance patterns across topics and model families, especially when ideological boundaries are ambiguous. These results position ControBench as a challenging and realistic benchmark for controversial discourse analysis.

Problem

• controversial discourse
• ideological polarization
• social interaction graphs
• online argumentation
• content moderation

Innovation

• controversial discourse analysis
• interaction-aware benchmark
• heterogeneous social graph
• ideological identity
• semantic-enriched edges