Algorithmic Behaviors Across Regions: A Geolocation Audit of YouTube Search for COVID-19 Misinformation between the United States and South Africa

2024-09-16

arXiv.org

AI Summary

This study identifies a systematic geographic bias in YouTube’s recommendation algorithm between the Global North and South. Through a 10-day geolocated audit comparing COVID-19 search results on YouTube’s homepage for users in the United States and South Africa, we find that 31.55% of top-ranked videos contain misinformation—and South African users are significantly more exposed to such content than their U.S. counterparts. Methodologically, we introduce the first dual-region comparative audit framework designed specifically for the Global South, integrating geo-fencing simulation, multi-location/multi-filter/multi-query sock-puppet automation, fact-check label classification, and rigorous statistical analysis. Our work fills a critical empirical gap in algorithmic auditing in low- and middle-income countries, providing the first reproducible validation of regionally contingent algorithmic bias. It delivers both foundational evidence and a methodological blueprint for global digital platform governance and algorithmic accountability.

Technology Category

• Search and Optimization: Algorithm Configuration
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Computer Vision: Bias, Fairness & Privacy

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Despite being an integral tool for finding health-related information online, YouTube has faced criticism for disseminating COVID-19 misinformation globally to its users. Yet, prior audit studies have predominantly investigated YouTube within the Global North contexts, often overlooking the Global South. To address this gap, we conducted a comprehensive 10-day geolocation-based audit on YouTube to compare the prevalence of COVID-19 misinformation in search results between the United States (US) and South Africa (SA), the countries heavily affected by the pandemic in the Global North and the Global South, respectively. For each country, we selected 3 geolocations and placed sock-puppets, or bots emulating"real"users, that collected search results for 48 search queries sorted by 4 search filters for 10 days, yielding a dataset of 915K results. We found that 31.55% of the top-10 search results contained COVID-19 misinformation. Among the top-10 search results, bots in SA faced significantly more misinformative search results than their US counterparts. Overall, our study highlights the contrasting algorithmic behaviors of YouTube search between two countries, underscoring the need for the platform to regulate algorithmic behavior consistently across different regions of the Globe.

Problem

• Compare COVID-19 misinformation prevalence in YouTube searches between US and South Africa
• Investigate algorithmic bias in YouTube search results across Global North and South
• Assess regional disparities in COVID-19 misinformation exposure via YouTube algorithms

Innovation

• Geolocation audit comparing US and South Africa
• Sock-puppet bots collecting 915K search results
• Analyzing YouTube's regional algorithmic disparities