Reducing Sexual Predation and Victimization Through Warnings and Awareness among High-Risk Users

2025-05-01

AI Summary

Online sexual predators progressively build relationships to perpetrate harm, while platforms typically respond only reactively after incidents occur. Method: Grounded in Routine Activity Theory, this study proposes a proactive human–AI collaborative intervention paradigm targeting the pre-offense behavioral stage. In the virtual social application *Pigg Party*, we integrate graph neural networks with multi-source behavioral features to build a real-time risk identification model. A large-scale randomized controlled trial delivered personalized warnings and awareness-enhancement messages to 12,842 high-risk users. Contribution/Results: This work pioneers the integration of Routine Activity Theory with platform-level dynamic risk modeling, transcending conventional content-moderation approaches. Results demonstrate statistically significant reductions in both female users’ rule-violating behaviors and victimization incidents, with effects sustained over 12 weeks—validating the efficacy and scalability of proactive, prevention-oriented online safety mechanisms.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Game Theory and Economic Paradigms: Behavioral Game Theory
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Mitigating online harms, including harassment, spam, and fake reviews
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Online sexual predators target children by building trust, creating dependency, and arranging meetings for sexual purposes. This poses a significant challenge for online communication platforms that strive to monitor and remove such content and terminate predators' accounts. However, these platforms can only take such actions if sexual predators explicitly violate the terms of service, not during the initial stages of relationship-building. This study designed and evaluated a strategy to prevent sexual predation and victimization by delivering warnings and raising awareness among high-risk individuals based on the routine activity theory in criminal psychology. We identified high-risk users as those with a high probability of committing or being subjected to violations, using a machine learning model that analyzed social networks and monitoring data from the platform. We conducted a randomized controlled trial on a Japanese avatar-based communication application, Pigg Party. High-risk players in the intervention group received warnings and awareness-building messages, while those in the control group did not receive the messages, regardless of their risk level. The trial involved 12,842 high-risk players in the intervention group and 12,844 in the control group for 138 days. The intervention successfully reduced violations and being violated among women for 12 weeks, although the impact on men was limited. These findings contribute to efforts to combat online sexual abuse and advance understanding of criminal psychology.

Problem

• Prevent online sexual predation via warnings and awareness for high-risk users
• Identify high-risk users using machine learning on social network data
• Evaluate intervention impact on reducing violations in online platforms

Innovation

• Machine learning identifies high-risk users
• Warnings and awareness messages reduce violations
• Randomized trial tests intervention effectiveness