ARCTraj: A Dataset and Benchmark of Human Reasoning Trajectories for Abstract Problem Solving

2025-11-14

AI Summary

Existing abstract reasoning research relies predominantly on static input-output pairs, failing to capture the dynamic, stepwise nature of human visual reasoning. Method: We introduce the first large-scale (≈10K trajectories), temporally ordered, object-level dataset of human problem-solving traces, covering 400 ARC-AGI-1 tasks; each trace is annotated with timestamps, operation types, and success labels. We propose a unified dynamic reasoning framework that reformulates static modeling as a Markov decision process (MDP), enabling seamless integration of sequential paradigms—including PPO, World Models, GFlowNets, diffusion agents, and Decision Transformers. Contribution/Results: Our dataset and framework uncover cognitive principles such as spatial selection bias, color-based inference patterns, and strategy convergence. They significantly advance interpretable, human-aligned reasoning modeling and establish a foundation for studying procedural abstraction in AI.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Visual Reasoning & Symbolic Representations

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We present ARCTraj, a dataset and methodological framework for modeling human reasoning through complex visual tasks in the Abstraction and Reasoning Corpus (ARC). While ARC has inspired extensive research on abstract reasoning, most existing approaches rely on static input--output supervision, which limits insight into how reasoning unfolds over time. ARCTraj addresses this gap by recording temporally ordered, object-level actions that capture how humans iteratively transform inputs into outputs, revealing intermediate reasoning steps that conventional datasets overlook. Collected via the O2ARC web interface, it contains around 10,000 trajectories annotated with task identifiers, timestamps, and success labels across 400 training tasks from the ARC-AGI-1 benchmark. It further defines a unified reasoning pipeline encompassing data collection, action abstraction, Markov decision process (MDP) formulation, and downstream learning, enabling integration with reinforcement learning, generative modeling, and sequence modeling methods such as PPO, World Models, GFlowNets, Diffusion agents, and Decision Transformers. Analyses of spatial selection, color attribution, and strategic convergence highlight the structure and diversity of human reasoning. Together, these contributions position ARCTraj as a structured and interpretable foundation for studying human-like reasoning, advancing explainability, alignment, and generalizable intelligence.

Problem

• Models human reasoning trajectories in abstract visual problem solving
• Captures intermediate reasoning steps overlooked by static datasets
• Provides structured framework for studying explainable and generalizable intelligence

Innovation

• Records temporally ordered object-level human reasoning actions
• Defines unified reasoning pipeline with MDP formulation
• Enables integration with reinforcement learning and generative modeling