Evaluation and Facilitation of Online Discussions in the LLM Era: A Survey

2025-03-03

AI Summary

In the LLM era, online discourse is increasingly plagued by hate speech proliferation and frequent breakdowns in mutual understanding—threatening social cohesion and democratic values. To address this, we develop an interdisciplinary analytical framework bridging NLP and social sciences. Methodologically, we propose a novel taxonomy for evaluating dialogue quality; introduce the first classification schema for dialogue guidance datasets encompassing multiple normative objectives; and design, for the first time, LLM-native dialogue guidance techniques—integrating human-AI collaborative modeling, prompt engineering, and targeted intervention strategies into a holistic “Assessment–Intervention–Governance” theoretical pipeline. Our core contribution lies in transcending isolated technical optimizations: we deliver a practical, implementation-ready framework for fostering rational, inclusive dialogue, alongside a transdisciplinary research paradigm that supports the development of resilient, equitable digital public spheres.

Technology Category

• Natural Language Processing: Conversational AI/Dialog Systems
• Machine Learning: Large Multimodal Models (LMMs)
• Application Domains: Humanities & Computational Social Science

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

We present a survey of methods for assessing and enhancing the quality of online discussions, focusing on the potential of Large Language Models (LLMs). While online discourses aim, at least in theory, to foster mutual understanding, they often devolve into harmful exchanges, such as hate speech, threatening social cohesion and democratic values. Recent advancements in LLMs enable facilitation agents that not only moderate content, but also actively improve the quality of interactions. Our survey synthesizes ideas from Natural Language Processing (NLP) and Social Sciences to provide (a) a new taxonomy on discussion quality evaluation, (b) an overview of intervention and facilitation strategies, along with a new taxonomy on conversation facilitation datasets, (c) an LLM-oriented roadmap of good practices and future research directions, from technological and societal perspectives.

Problem

• Assessing and enhancing online discussion quality using LLMs.
• Addressing harmful exchanges like hate speech in online discourses.
• Developing LLM-based facilitation strategies for better interactions.

Innovation

• LLMs enhance online discussion quality.
• New taxonomy for discussion evaluation.
• LLM-oriented roadmap for future research.