Catching Dark Signals in Algorithms: Unveiling Audiovisual and Thematic Markers of Unsafe Content Recommended for Children and Teenagers

2025-07-16

AI Summary

Short-video platforms expose children and adolescents to unsafe content due to ineffective age verification and opaque algorithmic recommendation systems. Method: We conducted a multimodal audit of 4,492 short videos from Instagram Reels, TikTok, and YouTube Shorts, integrating audiovisual deep representation learning, topic modeling, and algorithmic auditing to systematically identify both explicit harms (e.g., self-harm, violence) and implicit harms (e.g., negative emotion induction, identity-related anxiety). Contribution/Results: We propose a novel three-dimensional online harm framework—“explicit–implicit–unintended”—revealing cross-modal representational patterns and propagation logics of algorithmically amplified risks. Findings show implicit harms constitute a substantial, under-detected proportion of hazardous content, frequently evading current moderation systems. This study provides empirical evidence and theoretical tools to strengthen age-gating mechanisms, enhance platform algorithmic accountability, and inform evidence-based content governance policies.

Technology Category

• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data
• Application Domains: Humanities & Computational Social Science
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Mitigating online harms, including harassment, spam, and fake reviews
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

The prevalence of short form video platforms, combined with the ineffectiveness of age verification mechanisms, raises concerns about the potential harms facing children and teenagers in an algorithm-moderated online environment. We conducted multimodal feature analysis and thematic topic modeling of 4,492 short videos recommended to children and teenagers on Instagram Reels, TikTok, and YouTube Shorts, collected as a part of an algorithm auditing experiment. This feature-level and content-level analysis revealed that unsafe (i.e., problematic, mentally distressing) short videos (a) possess darker visual features and (b) contain explicitly harmful content and implicit harm from anxiety-inducing ordinary content. We introduce a useful framework of online harm (i.e., explicit, implicit, unintended), providing a unique lens for understanding the dynamic, multifaceted online risks facing children and teenagers. The findings highlight the importance of protecting younger audiences in critical developmental stages from both explicit and implicit risks on social media, calling for nuanced content moderation, age verification, and platform regulation.

Problem

• Detecting unsafe content in videos recommended to children
• Analyzing dark visual features and harmful themes in short videos
• Addressing explicit and implicit online risks for young audiences

Innovation

• Multimodal feature analysis for unsafe content detection
• Thematic topic modeling to identify harmful themes
• Framework categorizing explicit and implicit online harms