Detecting Struggling Student Programmers using Proficiency Taxonomies

2025-08-24

AI Summary

Early identification of struggling students in programming education remains challenging, and existing AI models lack explicit skill modeling. Method: This paper proposes an interpretable prediction framework grounded in an educator-coconstructed programming proficiency taxonomy. We embed this taxonomy into a deep learning model to jointly model coding behavior sequences and multi-task objectives—namely, fine-grained skill assessment and struggle prediction—thereby explicitly capturing students’ evolving competency trajectories during problem solving. Contribution/Results: To our knowledge, this is the first work to integrate teacher-informed skill structures directly into an end-to-end predictive model, achieving both high accuracy and intrinsic interpretability. Extensive experiments on Java and Python course datasets demonstrate statistically significant improvements over state-of-the-art baselines, with strong cross-language generalizability.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Structured Learning
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Early detection of struggling student programmers is crucial for providing them with personalized support. While multiple AI-based approaches have been proposed for this problem, they do not explicitly reason about students' programming skills in the model. This study addresses this gap by developing in collaboration with educators a taxonomy of proficiencies that categorizes how students solve coding tasks and is embedded in the detection model. Our model, termed the Proficiency Taxonomy Model (PTM), simultaneously learns the student's coding skills based on their coding history and predicts whether they will struggle on a new task. We extensively evaluated the effectiveness of the PTM model on two separate datasets from introductory Java and Python courses for beginner programmers. Experimental results demonstrate that PTM outperforms state-of-the-art models in predicting struggling students. The paper showcases the potential of combining structured insights from teachers for early identification of those needing assistance in learning to code.

Problem

• Early detection of struggling programming students
• Integrating proficiency taxonomies into AI models
• Predicting student struggle on new coding tasks

Innovation

• Embedded proficiency taxonomy in detection model
• Simultaneously learns skills and predicts struggles
• Combines teacher insights with AI for identification