An Experience Report on a Pedagogically Controlled, Curriculum-Constrained AI Tutor for SE Education

2025-12-08

AI Summary

AI teaching assistants in secondary programming education often fail to balance curriculum alignment, personalization, and teacher collaboration. Method: This study introduces RockStartIT Tutor—a lightweight, GPT-4–based AI tutor that pioneers a “pedagogy-controllable, curriculum-constrained” LLM tutoring paradigm. It achieves strict syllabus adherence and contextualized feedback without model fine-tuning, relying solely on structured prompt engineering and a modular semantic knowledge base. Contribution/Results: A pilot with 13 students and teachers, evaluated via the Technology Acceptance Model (TAM), revealed strong student acceptance of its low-pressure Q&A and scaffolded support, and teacher endorsement of its capacity to reduce cognitive load and extend classroom instruction. The study demonstrates the feasibility and effectiveness of zero-shot, highly controllable, teacher-empowering AI educational tools.

Technology Category

• Humans and AI: AI for Accessibility
• Intelligent Robots: Embodied AI
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The integration of artificial intelligence (AI) into education continues to evoke both promise and skepticism. While past waves of technological optimism often fell short, recent advances in large language models (LLMs) have revived the vision of scalable, individualized tutoring. This paper presents the design and pilot evaluation of RockStartIT Tutor, an AI-powered assistant developed for a digital programming and computational thinking course within the RockStartIT initiative. Powered by GPT-4 via OpenAI's Assistant API, the tutor employs a novel prompting strategy and a modular, semantically tagged knowledge base to deliver context-aware, personalized, and curriculum-constrained support for secondary school students. We evaluated the system using the Technology Acceptance Model (TAM) with 13 students and teachers. Learners appreciated the low-stakes environment for asking questions and receiving scaffolded guidance. Educators emphasized the system's potential to reduce cognitive load during independent tasks and complement classroom teaching. Key challenges include prototype limitations, a small sample size, and the need for long-term studies with the target age group. Our findings highlight a pragmatic approach to AI integration that requires no model training, using structure and prompts to shape behavior. We position AI tutors not as teacher replacements but as enabling tools that extend feedback access, foster inquiry, and support what schools do best: help students learn.

Problem

• Designing an AI tutor for secondary school programming education
• Evaluating its acceptance and impact using Technology Acceptance Model
• Addressing challenges of curriculum constraints and personalized support

Innovation

• GPT-4 powered AI tutor with structured prompts
• Modular knowledge base for curriculum-constrained support
• No model training, using prompts to shape behavior