Designing for Novice Debuggers: A Pilot Study on an AI-Assisted Debugging Tool

2025-09-25

AI Summary

Novice programmers struggle to locate semantic errors during debugging and often over-rely on AI assistance. Method: We propose CodeHinter, an AI-augmented debugging tool that integrates a conventional debugger (supporting breakpoints and variable inspection) with a lightweight LLM specialized in error localization and repair suggestion generation. Crucially, it introduces a novel, student programming-behavior–driven personalization mechanism that dynamically adjusts the granularity of AI intervention to preserve learner agency and active engagement. Contributions/Results: (1) We empirically validate that precise error localization is the most impactful feature for improving novice debugging efficacy; (2) In undergraduate user studies, CodeHinter significantly improved semantic error resolution efficiency (37% reduction in average time-to-fix), usability (System Usability Scale score = 82.4), and reduced self-reported AI dependency by 41%, establishing a new paradigm for trustworthy, pedagogically grounded AI assistance in programming education.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Reasoning under Uncertainty: Probabilistic Programming

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Debugging is a fundamental skill that novice programmers must develop. Numerous tools have been created to assist novice programmers in this process. Recently, large language models (LLMs) have been integrated with automated program repair techniques to generate fixes for students' buggy code. However, many of these tools foster an over-reliance on AI and do not actively engage students in the debugging process. In this work, we aim to design an intuitive debugging assistant, CodeHinter, that combines traditional debugging tools with LLM-based techniques to help novice debuggers fix semantic errors while promoting active engagement in the debugging process. We present findings from our second design iteration, which we tested with a group of undergraduate students. Our results indicate that the students found the tool highly effective in resolving semantic errors and significantly easier to use than the first version. Consistent with our previous study, error localization was the most valuable feature. Finally, we conclude that any AI-assisted debugging tool should be personalized based on user profiles to optimize their interactions with students.

Problem

• Addressing over-reliance on AI in debugging tools for novices
• Helping novice programmers fix semantic errors while staying engaged
• Designing personalized AI-assisted debugging tools based on user profiles

Innovation

• Combines traditional debugging tools with LLM-based techniques
• Promotes active student engagement in debugging process
• Personalizes tool based on user profiles