CAREBench: A Child-Safety Risk Benchmark for Language Models

2026-06-28

AI Summary

This work addresses a critical gap in current language model safety evaluations, which predominantly focus on explicit child sexual abuse material while overlooking upstream risks such as adult grooming, emotional manipulation, AI anthropomorphism, and the induction of emotional dependency in minors. To bridge this gap, the authors introduce CAREBench—the first comprehensive benchmark systematically covering twelve categories of upstream child safety risks—comprising 500 non-explicit harmful prompts annotated by parents and clinical experts. Using multi-category prompt engineering and a fine-grained model response taxonomy (identify, refuse, mitigate, or redirect), experiments across seven state-of-the-art language models reveal failure rates ranging from 2% to 58%, highlighting substantial disparities in their preventive safeguarding capabilities and providing an empirical foundation for improving child safety alignment strategies.

Abstract

How can we evaluate whether frontier AI systems recognize child-safety risks before they escalate into explicit harm? Existing child safety evaluations focus on child sexual abuse material, yet many child-safety failures begin earlier: in model assistance that helps adults manipulate, impersonate, profile, or isolate minors, and in model responses that deepen children's emotional dependence on AI systems rather than redirecting them toward human support. We introduce CAREBench (Child AI Risk Evaluation), a benchmark to assess such upstream child-safety risks in language models. CAREBench contains 500 prompts spanning twelve risk categories, including grooming and relationship engineering, deception and impersonation, surveillance and privacy, sextortion and sexual abuse, AI anthropomorphization, emotional dependency, and mental illness sensitivity. Developed with response annotations from parents and clinicians, the benchmark excludes explicit abuse material and imagery; instead, it evaluates whether models recognize, refuse, de-escalate, or redirect risky interactions before harm becomes overt. Evaluating seven frontier models on our benchmark, we find failure rates ranging from 2% to 58%, with failure patterns that vary across risk categories. CAREBench provides a responsibly scoped evaluation for LLM developers to identify and close gaps in child safety policies.

Problem

• child safety
• language models
• risk evaluation
• emotional dependency
• grooming

Innovation

• child safety
• risk benchmark
• language models
• upstream risk detection
• AI ethics