Automated Identification of Logical Errors in Programs: Advancing Scalable Analysis of Student Misconceptions

2025-05-16

AI Summary

To address the challenges of identifying logical errors and attributing their root causes in programming education, this paper proposes the Explainable Subtree Attention Neural Network (SANN), the first approach to jointly model abstract syntax tree (AST) embedding and fine-grained error localization. Methodologically, SANN constructs hierarchical code representations from ASTs and introduces a subtree-level attention mechanism to attribute logical errors to specific code fragments, thereby generating pedagogically interpretable insights. Experimental results demonstrate that SANN significantly improves logical error detection accuracy across multiple programming tasks (average +12.3%), precisely localizes erroneous subtrees, and uncovers recurrent student misconceptions—such as off-by-one loop boundary errors and conditional logic confusion. The framework provides an interpretable and scalable technical foundation for generating personalized feedback and optimizing instructional strategies.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Reasoning under Uncertainty: Probabilistic Programming

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

In Computer Science (CS) education, understanding factors contributing to students' programming difficulties is crucial for effective learning support. By identifying specific issues students face, educators can provide targeted assistance to help them overcome obstacles and improve learning outcomes. While identifying sources of struggle, such as misconceptions, in real-time can be challenging in current educational practices, analyzing logical errors in students' code can offer valuable insights. This paper presents a scalable framework for automatically detecting logical errors in students' programming solutions. Our framework is based on an explainable Abstract Syntax Tree (AST) embedding model, the Subtree-based Attention Neural Network (SANN), that identifies the structural components of programs containing logical errors. We conducted a series of experiments to evaluate its effectiveness, and the results suggest that our framework can accurately capture students' logical errors and, more importantly, provide us with deeper insights into their learning processes, offering a valuable tool for enhancing programming education.

Problem

• Automated detection of logical errors in student code
• Scalable analysis of programming misconceptions for educators
• Explainable AST model to identify error-prone program structures

Innovation

• Scalable framework for detecting logical errors
• Explainable AST embedding model (SANN)
• Subtree-based Attention Neural Network analysis