Enhancing Privacy in the Early Detection of Sexual Predators Through Federated Learning and Differential Privacy

2025-01-21

AI Summary

To address the challenge of balancing privacy preservation and model utility in detecting online grooming targeting children, this paper proposes the first localized detection framework that deeply integrates federated learning with differential privacy. The framework eliminates the need to upload or centrally store sensitive chat data, enabling on-device model training and privacy-enhanced inference. Innovatively, gradient clipping and calibrated noise injection are embedded directly into the federated aggregation process, ensuring end-to-end privacy guarantees. Evaluated on a real-world dataset of child–adult conversations, the framework achieves an F1-score of 89.7% and only a 1.3% accuracy drop under a stringent privacy budget of ε = 2—substantially outperforming baseline approaches. Results demonstrate that the method attains deployable-level detection performance under strong privacy constraints, establishing a new paradigm for child online protection that simultaneously satisfies regulatory compliance and practical effectiveness.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Safety and Robustness
• Computer Vision: Bias, Fairness & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Security and Privacy: Privacy-enhancing technologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The increased screen time and isolation caused by the COVID-19 pandemic have led to a significant surge in cases of online grooming, which is the use of strategies by predators to lure children into sexual exploitation. Previous efforts to detect grooming in industry and academia have involved accessing and monitoring private conversations through centrally-trained models or sending private conversations to a global server. In this work, we implement a privacy-preserving pipeline for the early detection of sexual predators. We leverage federated learning and differential privacy in order to create safer online spaces for children while respecting their privacy. We investigate various privacy-preserving implementations and discuss their benefits and shortcomings. Our extensive evaluation using real-world data proves that privacy and utility can coexist with only a slight reduction in utility.

Problem

• Child Online Protection
• Privacy Preservation
• Predatory Behavior Detection

Innovation

• Federated Learning
• Differential Privacy
• Child Online Protection