Revisiting Noise in Natural Language Processing for Computational Social Science

2025-03-10

AI Summary

This study addresses multi-source noise in computational social science (CSS)—including OCR errors, archaic linguistic variants, subjective annotation inconsistencies, and large language model (LLM) generation biases—challenging the conventional view that noise is merely disruptive. Method: We propose a novel, context-aware, typology-driven noise governance framework integrating text robustness analysis, historical language modeling, annotation consistency evaluation, and interpretable LLM bias diagnostics. Contribution/Results: Empirical analysis identifies four canonical noise types and demonstrates their substantive information value—encoding sociocultural patterns and individual behavioral signals. The framework provides theoretical grounding and actionable strategies for CSS task design, data curation, and model evaluation, reframing noise not as expendable artifact but as analyzable social evidence.

Technology Category

• Application Domains: Humanities & Computational Social Science
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Social Networks and Social Media: Computational social science
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Computational Social Science (CSS) is an emerging field driven by the unprecedented availability of human-generated content for researchers. This field, however, presents a unique set of challenges due to the nature of the theories and datasets it explores, including highly subjective tasks and complex, unstructured textual corpora. Among these challenges, one of the less well-studied topics is the pervasive presence of noise. This thesis aims to address this gap in the literature by presenting a series of interconnected case studies that examine different manifestations of noise in CSS. These include character-level errors following the OCR processing of historical records, archaic language, inconsistencies in annotations for subjective and ambiguous tasks, and even noise and biases introduced by large language models during content generation. This thesis challenges the conventional notion that noise in CSS is inherently harmful or useless. Rather, it argues that certain forms of noise can encode meaningful information that is invaluable for advancing CSS research, such as the unique communication styles of individuals or the culture-dependent nature of datasets and tasks. Further, this thesis highlights the importance of nuance in dealing with noise and the considerations CSS researchers must address when encountering it, demonstrating that different types of noise require distinct strategies.

Problem

• Addressing noise in Computational Social Science datasets.
• Exploring noise as meaningful information in CSS research.
• Developing strategies for handling diverse types of noise.

Innovation

• Analyzes noise in computational social science
• Explores noise as meaningful information source
• Proposes distinct strategies for noise types