ChildEval: When large language models meet children's personalities

2026-05-26

AI Summary

This work addresses the lack of systematic evaluation of large language models (LLMs) with respect to children’s personalized preferences. To this end, we introduce ChildEval—the first fine-grained preference benchmark tailored for children aged 3–6—comprising 29K synthetically generated character profiles that encode both explicit and implicit preference expressions. We further design a dedicated evaluation protocol to assess an LLM’s ability to understand and consistently adhere to child-centered preferences throughout extended dialogues. Leveraging synthetic data generation, long-context modeling, and preference alignment strategies, we fine-tune and evaluate open-source LLMs, examining how different representations of personalization influence response quality. Experimental results demonstrate that fine-tuning on ChildEval significantly enhances model performance on child-centered tasks.

Abstract

While LLMs enable personalized chatbots, their effectiveness in child-centered personalization remains unclear, as systematic evaluation of child-specific preferences is still lacking. To address this gap, we introduce ChildEval, a benchmark for evaluating LLMs' ability to infer and follow child-centered preferences in long-context conversations. ChildEval contains 29K synthesized persona profiles of children aged 3-6, providing relatively static background information. Each persona is associated with a child preference-which may align with, conflict with, or be independent of the persona-expressed either explicitly in a single sentence or implicitly through 6-10 turn dialogues. Explicit and implicit preferences are designed to reflect the same underlying preference but differ in expression, capturing dynamic aspects of preference expression rather than changes in the static persona. The benchmark spans five top-level and fourteen sub-level categories covering children's daily lives and development. We further propose fine-grained, child-centric evaluation protocols to systematically assess open-source LLMs. Experimental results demonstrate how different personalized representations affect LLM responses and suggest that finetuning on ChildEval can enhance child-centered performance. Our code and dataset are available at https://github.com/ziyanluo/ChildEval.

Problem

• large language models
• child-centered personalization
• preference evaluation
• children's personalities
• systematic benchmarking

Innovation

• ChildEval
• child-centered personalization
• preference inference
• long-context dialogue
• synthetic persona