SPOT: An Annotated French Corpus and Benchmark for Detecting Critical Interventions in Online Conversations

2025-11-10

AI Summary

This work formally defines the sociological concept of “stopping point”—a discourse-interrupting or redirection-triggering utterance (e.g., irony, subtle skepticism, fragmented arguments)—as a novel, computationally tractable NLP task for French social media. Such interventions are routinely overlooked by conventional adversarial discourse detection frameworks. To support this task, we introduce SPOT, the first manually annotated French corpus featuring multi-level contextual grounding (post, parent comment, and source article). We systematically evaluate both fine-tuned encoder models (e.g., CamemBERT) and instruction-tuned large language models under diverse prompting strategies. Supervised fine-tuning achieves an F1-score of 0.78—outperforming the best prompt-based approach by over ten percentage points. This study bridges a critical gap in fine-grained critical discourse analysis beyond English, demonstrating the empirical necessity of structured annotation and supervised learning for emerging socio-semantic NLP tasks.

Technology Category

• Natural Language Processing: Discourse, Pragmatics & Argument Mining
• Application Domains: Humanities & Computational Social Science
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Web Mining and Content Analysis: Sentiment analysis and opinion mining
• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

We introduce SPOT (Stopping Points in Online Threads), the first annotated corpus translating the sociological concept of stopping point into a reproducible NLP task. Stopping points are ordinary critical interventions that pause or redirect online discussions through a range of forms (irony, subtle doubt or fragmentary arguments) that frameworks like counterspeech or social correction often overlook. We operationalize this concept as a binary classification task and provide reliable annotation guidelines. The corpus contains 43,305 manually annotated French Facebook comments linked to URLs flagged as false information by social media users, enriched with contextual metadata (article, post, parent comment, page or group, and source). We benchmark fine-tuned encoder models (CamemBERT) and instruction-tuned LLMs under various prompting strategies. Results show that fine-tuned encoders outperform prompted LLMs in F1 score by more than 10 percentage points, confirming the importance of supervised learning for emerging non-English social media tasks. Incorporating contextual metadata further improves encoder models F1 scores from 0.75 to 0.78. We release the anonymized dataset, along with the annotation guidelines and code in our code repository, to foster transparency and reproducible research.

Problem

• Detecting subtle critical interventions in French online conversations
• Creating benchmark for non-English social media content moderation
• Improving classification of overlooked discussion-stopping rhetorical forms

Innovation

• Binary classification task for detecting stopping points
• Fine-tuned CamemBERT encoder models outperform prompted LLMs
• Incorporating contextual metadata improves model F1 scores