Understanding Parents' Desires in Moderating Children's Interactions with GenAI Chatbots through LLM-Generated Probes

2026-03-04

AI Summary

This study addresses the lack of fine-grained parental oversight mechanisms in current generative AI chatbots for children. It innovatively leverages large language models (LLMs) to generate realistic child–AI dialogues as probes, integrating parental feedback, human validation, and qualitative interviews to systematically investigate requirements for content moderation, transparency, and personalized control. Moving beyond existing coarse-grained approaches, the research reveals three key insights: parents are deeply concerned about latent risk scenarios overlooked by current tools, demand transparency and intervention capabilities at the dialogue level, and emphasize the need for age-appropriate, customizable guardianship strategies. This work provides foundational design principles and methodological support for developing trustworthy AI interaction systems tailored to family contexts.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation
• Humans and AI: Game Design — Procedural Content Generation & Storytelling

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

This paper studies how parents want to moderate children's interactions with Generative AI chatbots, with the goal of informing the design of future GenAI parental control tools. We first used an LLM to generate synthetic child-GenAI chatbot interaction scenarios and worked with four parents to validate their realism. From this dataset, we carefully selected 12 diverse examples that evoked varying levels of concern and were rated the most realistic. Each example included a prompt and a GenAI chatbot response. We presented these to parents (N=24) and asked whether they found them concerning, why, and how they would prefer the responses to be modified and communicated. Our findings reveal three key insights: (1) parents express concern about interactions that current GenAI chatbot parental controls neglect; (2) parents want fine-grained transparency and moderation at the conversation level; and (3) parents need personalized controls that adapt to their desired strategies and children's ages.

Problem

• parental control
• Generative AI
• child-AI interaction
• moderation
• LLM

Innovation

• LLM-generated probes
• parental control
• GenAI chatbots
• fine-grained moderation
• personalized AI governance