A Multi-Labeled Dataset for Indonesian Discourse: Examining Toxicity, Polarization, and Demographics Information

2025-03-01

AI Summary

Rising political polarization in Indonesia has intensified online toxicity toward minority groups, yet existing NLP research lacks systematic investigation into the interplay among toxicity, political polarization, and demographic factors. Method: We introduce the first Indonesian multilabel dataset, jointly annotating texts for toxicity severity, political polarization stance, and annotator demographics (e.g., gender, education level), enabling the first unified modeling of these three dimensions. Our approach proposes a multitask learning framework integrating label-cooperative learning and demographic-aware representation, evaluated on BERT-base and LLM baselines. Contribution/Results: Incorporating polarization signals improves toxicity detection F1 by 4.2%; integrating demographic features boosts polarization classification accuracy by 7.8%. Empirical results demonstrate that multidimensional annotation significantly enhances model robustness and fairness, offering a novel paradigm for analyzing structural biases in digital public spheres.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Ethics, Bias, and Fairness
• Application Domains: Humanities & Computational Social Science

Application Category

• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Polarization is defined as divisive opinions held by two or more groups on substantive issues. As the world's third-largest democracy, Indonesia faces growing concerns about the interplay between political polarization and online toxicity, which is often directed at vulnerable minority groups. Despite the importance of this issue, previous NLP research has not fully explored the relationship between toxicity and polarization. To bridge this gap, we present a novel multi-label Indonesian dataset that incorporates toxicity, polarization, and annotator demographic information. Benchmarking this dataset using BERT-base models and large language models (LLMs) shows that polarization information enhances toxicity classification, and vice versa. Furthermore, providing demographic information significantly improves the performance of polarization classification.

Problem

• Explores the relationship between online toxicity and political polarization in Indonesia.
• Introduces a multi-label dataset combining toxicity, polarization, and demographic data.
• Demonstrates that polarization and demographic data improve toxicity and polarization classification.

Innovation

• Developed multi-label Indonesian dataset
• Used BERT-base and LLMs for benchmarking
• Incorporated demographic data for classification