Children's Intelligence Tests Pose Challenges for MLLMs? KidGym: A 2D Grid-Based Reasoning Benchmark for MLLMs

2026-03-02

AI Summary

Current multimodal large language models (MLLMs) lack a systematic evaluation benchmark tailored to child-like cognitive capabilities, making it difficult to assess their developmental progress across core dimensions of intelligence such as execution, perceptual reasoning, learning, memory, and planning. Inspired by the Wechsler Intelligence Scale for Children, this work proposes KidGym—a customizable and scalable 2D grid-based benchmark comprising twelve tasks that span these five cognitive domains. Leveraging procedural generation, KidGym produces diverse scenarios with dynamically adjustable difficulty, enabling fine-grained and interpretable multidimensional assessment. Experimental results reveal significant deficiencies in existing MLLMs, particularly in memory retention, planning coherence, and cross-task generalization, thereby offering clear directions for future model improvement.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Multimodal Large Language Models (MLLMs) combine the linguistic strengths of LLMs with the ability to process multimodal data, enbaling them to address a broader range of visual tasks. Because MLLMs aim at more general, human-like competence than language-only models, we take inspiration from the Wechsler Intelligence Scales - an established battery for evaluating children by decomposing intelligence into interpretable, testable abilities. We introduce KidGym, a comprehensive 2D grid-based benchmark for assessing five essential capabilities of MLLMs: Execution, Perception Reasoning, Learning, Memory and Planning. The benchmark comprises 12 unique tasks, each targeting at least one core capability, specifically designed to guage MLLMs'adaptability and developmental potential, mirroring the stages of children's cognitive growth. Additionally, our tasks encompass diverse scenarios and objects with randomly generated layouts, ensuring a more accurate and robust evluation of MLLM capabilities. KidGym is designed to be fully user-customizable and extensible, allowing researchers to create new evaluation scenarios and adjust difficuly levels to accommodate the rapidly growing MLLM community. Through the evaluation of state-of-the-art MLLMs using KidGym, we identified significant insights into model capabilities and revealed several limitations of current models. We release our benchmark at: https://kidgym.github.io/KidGym-Website/.

Problem

• Multimodal Large Language Models
• Intelligence Evaluation
• 2D Grid-Based Reasoning
• Cognitive Benchmark
• Children's Intelligence

Innovation

• Multimodal Large Language Models
• Cognitive Benchmarking
• 2D Grid-based Reasoning
• Child-inspired AI Evaluation
• Extensible Benchmark