Learning to Make MISTAKEs: Modeling Incorrect Student Thinking And Key Errors

2025-10-13

AI Summary

To address the need for precise simulation of students’ erroneous reasoning patterns in educational settings, this paper proposes MISTAKE—a method that leverages cycle-consistent modeling to establish bidirectional mappings between incorrect answers and underlying misconceptions, enabling automated synthesis of high-quality erroneous reasoning data. MISTAKE integrates misconception classification, incorrect answer generation, and student behavior simulation, eliminating reliance on manual annotation. Evaluated on three educational tasks, it significantly improves error simulation accuracy (+12.3%), enhances the ability to infer deep misconceptions from incorrect answers (F1 +15.6%), and generates distractors of expert-level quality (89.4% agreement with domain experts). This work introduces cycle consistency to educational error modeling for the first time, delivering a scalable, interpretable framework for generating reasoning errors—thereby advancing intelligent tutoring feedback and teacher training systems.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Research on reasoning in language models (LMs) predominantly focuses on improving the correctness of their outputs. But some important applications require modeling reasoning patterns that are incorrect. For example, automated systems that can reason about and simulate student errors are useful for providing real-time feedback in the classroom or offline practice for educators-in-training. This paper presents a new method, MISTAKE, that (1) constructs high-quality synthetic examples of reasoning errors by leveraging cycle consistency between incorrect answers and latent misconceptions; and (2) uses the generated data to learn models for student simulation, misconception classification, and answer generation. We evaluate MISTAKE on three educational tasks and find that it results in (1) higher accuracy when simulating incorrect student answers based on specific misconceptions, (2) increased performance inferring latent misconceptions from observed incorrect answers, and (3) higher alignment with expert-written distractor answers when generating incorrect answers (e.g., for multiple-choice tests).

Problem

• Modeling incorrect student thinking and key errors
• Simulating student errors for real-time feedback
• Generating synthetic examples of reasoning errors

Innovation

• Generates synthetic reasoning errors using cycle consistency
• Learns models for student simulation and misconception classification
• Improves accuracy in simulating incorrect student answers