SketchMind: A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketches

2025-06-29

arXiv.org

AI Summary

This study addresses the challenge of automating the assessment of students’ freehand sketches—such as conceptual models—in science education. We propose the first assessment framework integrating cognitive theory with multi-agent collaboration. The framework employs modular, cooperative agents to perform rubric parsing, multi-granularity image understanding, dynamic alignment with Bloom’s taxonomy, and structured reasoning-guided (SRG) iterative feedback—ensuring interpretability, pedagogical alignment, and adaptability across cognitive levels. Evaluated on 3,575 authentic student sketches, our approach achieves a mean assessment accuracy of 77.1%, outperforming baselines by 21.4%; domain experts rate it 4.1/5, significantly surpassing existing methods. Our core contribution is a novel multi-agent sketch assessment paradigm that is cognitively traceable, pedagogically embeddable, and feedback-directable.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Modeling other Agents
• Knowledge Representation and Reasoning: Argumentation

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Scientific sketches (e.g., models) offer a powerful lens into students'conceptual understanding, yet AI-powered automated assessment of such free-form, visually diverse artifacts remains a critical challenge. Existing solutions often treat sketch evaluation as either an image classification task or monolithic vision-language models, which lack interpretability, pedagogical alignment, and adaptability across cognitive levels. To address these limitations, we present SketchMind, a cognitively grounded, multi-agent framework for evaluating and improving student-drawn scientific sketches. SketchMind comprises modular agents responsible for rubric parsing, sketch perception, cognitive alignment, and iterative feedback with sketch modification, enabling personalized and transparent evaluation. We evaluate SketchMind on a curated dataset of 3,575 student-generated sketches across six science assessment items with different highest order of Bloom's level that require students to draw models to explain phenomena. Compared to baseline GPT-4o performance without SRG (average accuracy: 55.6%), and with SRG integration achieves 77.1% average accuracy (+21.4% average absolute gain). We also demonstrate that multi-agent orchestration with SRG enhances SketchMind performance, for example, GPT-4.1 gains an average 8.9% increase in sketch prediction accuracy, outperforming single-agent pipelines across all items. Human evaluators rated the feedback and co-created sketches generated by extsc{SketchMind} with GPT-4.1, which achieved an average of 4.1 out of 5, significantly higher than those of baseline models (e.g., 2.3 for GPT-4o). Experts noted the system's potential to meaningfully support conceptual growth through guided revision. Our code and (pending approval) dataset will be released to support reproducibility and future research in AI-driven education.

Problem

• Automated assessment of student-drawn scientific sketches remains challenging
• Existing solutions lack interpretability and pedagogical alignment across cognitive levels
• Current methods struggle with free-form visually diverse sketch artifacts

Innovation

• Multi-agent cognitive framework for sketch assessment
• Modular agents enable transparent personalized evaluation
• Orchestrated agents outperform single-agent pipelines significantly