White Paper: Human-AI Collaboration in Conflict Analysis: Text Classifier Development with Peacebuilders

2026-04-22

AI Summary

This study addresses the challenges of misclassification and limited local practitioner buy-in in online polarization and hate speech monitoring, stemming from cultural contextual differences. To tackle these issues, the project collaborates with peacebuilders and data scientists from Kenya and Sudan through a participatory annotation process to jointly define tasks, design labeling schemes, and iteratively validate models. The methodology deeply integrates domain experts throughout the AI development lifecycle, combining BERT-based fine-tuned classifiers with a cross-cultural contextual validation mechanism. The resulting open-source classifiers—Kenya-polarization and Sudan-hate-speech—released on Hugging Face, demonstrate significantly improved classification accuracy under culturally sensitive conditions and greater local acceptance of the tools, achieving synergistic optimization of technical robustness, contextual validity, and ethical alignment.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Application Domains: Humanities & Computational Social Science

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Responsible Web: Mitigating online harms, including harassment, spam, and fake reviews

Abstract

This paper documents a collaborative research process involving peacebuilders and data scientists in Kenya and Sudan to develop AI-based text classifiers for monitoring online polarization and hatespeech. The method describes a participatory annotation process in which practitioners and domain experts contributed to problem definition, annotation design, iterative validation, and model evaluation. Fine-tuned BERT-based classifiers were trained on collaboratively annotated datasets and evaluated against held-out test sets. In each case, the models produced enhanced contextual alignment, reduced misclassification driven by cultural nuance, and increased practitioner ownership of AI tools. The resulting models (Kenya-polarization and Sudan-hate speech) are open-source and accessible via HuggingFace. The study contributes empirical evidence that participatory AI development can simultaneously improve technical robustness, contextual validity, and normative alignment in sensitive humanitarian domains.

Problem

• human-AI collaboration
• conflict analysis
• text classification
• online polarization
• hate speech

Innovation

• participatory AI
• human-AI collaboration
• contextual alignment
• culturally-aware NLP
• fine-tuned BERT