Multimodal Programming in Computer Science with Interactive Assistance Powered by Large Language Model

2025-03-09

AI Summary

This work addresses the misalignment of large language model (LLM)-based teaching assistants with pedagogical objectives in introductory programming education. We propose an education-goal-driven prompt encapsulation mechanism and implement an interactive programming tutor grounded in DeepSeek-R1, integrated into both IDEs and command-line assessment tools. The system performs fine-grained error diagnosis and delivers tiered, pedagogically scaffolded feedback—including conceptual hints, solution planning guidance, and template code—while deliberately withholding complete solutions to foster conceptual understanding. Empirical evaluation demonstrates that the tutor effectively identifies students’ conceptual misconceptions and delivers instructionally appropriate feedback. However, it also reveals critical alignment challenges, including erroneous rejection of correct code and recommendation of pedagogically mismatched solutions. To our knowledge, this is the first study to explicitly embed educational goals into LLM prompt design, offering a reproducible methodological framework and empirical evidence for improving pedagogical alignment in AI teaching assistants.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Reasoning under Uncertainty: Probabilistic Programming

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

LLM chatbot interfaces allow students to get instant, interactive assistance with homework, but doing so carelessly may not advance educational objectives. In this study, an interactive homework help system based on DeepSeek R1 is developed and first implemented for students enrolled in a large computer science beginning programming course. In addition to an assist button in a well-known code editor, our assistant also has a feedback option in our command-line automatic evaluator. It wraps student work in a personalized prompt that advances our educational objectives without offering answers straight away. We have discovered that our assistant can recognize students' conceptual difficulties and provide ideas, plans, and template code in pedagogically appropriate ways. However, among other mistakes, it occasionally incorrectly labels the correct student code as incorrect or encourages students to use correct-but-lesson-inappropriate approaches, which can lead to long and frustrating journeys for the students. After discussing many development and deployment issues, we provide our conclusions and future actions.

Problem

• Develops an interactive homework help system for programming students.
• Addresses conceptual difficulties with personalized, pedagogically appropriate assistance.
• Identifies and mitigates errors in code evaluation and feedback.

Innovation

• Interactive homework help system using DeepSeek R1
• Personalized prompts for educational advancement
• Feedback option in command-line evaluator