An Online Integrated Development Environment for Automated Programming Assessment Systems

2025-03-17

AI Summary

Current automatic programming assessment systems (APASs) in programming education lack core IDE functionalities—such as syntax highlighting and intelligent code completion—degrading pedagogical effectiveness and learner engagement. To address this gap, this work formally defines the functional requirements for educational online IDEs and designs, implements, and open-sources a lightweight, secure, and extensible web-based prototype. The system integrates real-time syntax highlighting, context-aware code completion, and automated assessment capabilities, balancing pedagogical usability with computational efficiency. Usability is evaluated via the Technology Acceptance Model (TAM), while performance is quantified using metrics including RAM utilization and response latency. An empirical study with 27 first-year computer science students demonstrates high acceptance, low memory footprint (<150 MB), sub-200 ms average response time, and strong scalability. This constitutes the first open-source, APAS-optimized online IDE specifically designed for programming education.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Application Domains: Humanities & Computational Social Science

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

The increasing demand for programmers has led to a surge in participants in programming courses, making it increasingly challenging for instructors to assess student code manually. As a result, automated programming assessment systems (APASs) have been developed to streamline this process. These APASs support lecturers by managing and evaluating student programming exercises at scale. However, these tools often do not provide feature-rich online editors compared to their traditional integrated development environments (IDEs) counterparts. This absence of key features, such as syntax highlighting and autocompletion, can negatively impact the learning experience, as these tools are crucial for effective coding practice. To address this gap, this research contributes to the field of programming education by extracting and defining requirements for an online IDE in an educational context and presenting a prototypical implementation of an open-source solution for a scalable and secure online IDE. The usability of the new online IDE was assessed using the Technology Acceptance Model (TAM), gathering feedback from 27 first-year students through a structured survey. In addition to these qualitative insights, quantitative measures such as memory (RAM) usage were evaluated to determine the efficiency and scalability of the tool under varying usage conditions.

Problem

• Addresses lack of feature-rich online IDEs in automated programming assessment systems.
• Proposes scalable, secure online IDE for programming education.
• Evaluates usability and efficiency of new online IDE using TAM.

Innovation

• Developed scalable online IDE for programming education
• Integrated syntax highlighting and autocompletion features
• Assessed usability using Technology Acceptance Model