Spark: Real-Time Monitoring of Multi-Faceted Programming Exercises

2025-10-07

IEEE Symposium on Visual Languages / Human-Centric Computing Languages and Environments

AI Summary

This work addresses the challenge of real-time monitoring of student progress in complex, nonlinear programming exercises, particularly when task steps lack a fixed sequence or consistent evaluation criteria. To this end, the authors propose SPARK, a monitoring dashboard that, for the first time, enables instructors to flexibly define custom checkpoints. By leveraging keystroke-level coding data, SPARK automatically generates tests and dynamically groups students, facilitating fine-grained, configurable, and real-time progress tracking. The system employs interactive visualizations to reveal students’ intermediate outputs and the diversity of their problem-solving pathways, thereby supporting instructors in gaining deeper insights into individual learning differences. Empirical evaluation involving 22 learners and 16 instructors demonstrates that SPARK significantly enhances instructional awareness and effectively identifies student difficulties.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Reasoning under Uncertainty: Probabilistic Programming

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

Monitoring in-class programming exercises can help instructors identify struggling students and common challenges. However, understanding students’ progress can be prohibitively difficult, particularly for multi-faceted problems that include multiple steps with complex interdependencies, have no predictable completion order, or involve evaluation criteria that are difficult to summarize across many students (e.g., exercises building interactive web-based user interfaces). We introduce Spark, a coding exercise monitoring dashboard designed to address these challenges. Spark allows instructors to flexibly group substeps into checkpoints based on exercise requirements, suggests automated tests for these checkpoints, and generates visualizations to track progress across steps. Spark also allows instructors to inspect intermediate outputs, providing deeper insights into solution variations. We also construct a dataset of 40 -minute keystroke coding data from $\mathrm{N}=22$ learners solving two web programming exercises and provide empirical insights into the perceived usefulness of Spark through a within-subjects evaluation with $\mathbf{1 6}$ programming instructors. Index Terms-programming education

Problem

• real-time monitoring
• multi-faceted programming exercises
• student progress tracking
• in-class programming
• complex interdependencies

Innovation

• real-time monitoring
• multi-faceted programming exercises
• checkpoint-based assessment
• automated test suggestion
• interactive visualization