ProDebug: An Automated Debugging System for Prolog

2026-05-26

AI Summary

This work addresses the lack of effective debugging mechanisms for beginners in Prolog, which hinders the scalability of logic programming education. It proposes the first automated debugging and repair approach tailored to the declarative language Prolog by integrating large language models (LLMs) with spectrum-based fault localization and mutation testing. The method leverages student code submissions and Git commit histories to automatically identify bugs and generate targeted repair suggestions. Evaluated on 1,499 real student assignments, the system demonstrates high efficacy in fault detection and repair generation, significantly enhancing the capacity for automated, personalized feedback in Prolog programming instruction.

Abstract

Prolog is a well-known declarative programming language commonly used in introductory courses on logic and reasoning. However, many students find Prolog challenging because it lacks the familiar debugging mechanisms found in imperative languages. In large classes, this difficulty is exacerbated by the challenge of providing timely and personalized feedback to students. In this work, we introduce ProDebug, the first tool to combine Large Language Models (LLMs) with spectrum-based and mutation-based techniques for automated debugging of Prolog assignments. ProDebug automatically identifies faults and proposes bug repairs for student Git submissions. Faults are detected using three approaches--spectrum-based, mutation-based, and LLM reasoning--while repairs are generated using mutation-based techniques and LLMs. Our evaluation on 1499 buggy student submissions from a bachelor's level programming class demonstrates the potential of automated, LLM-augmented feedback systems to scale support for declarative programming education.

Problem

• Prolog
• debugging
• automated feedback
• declarative programming
• student submissions

Innovation

• Large Language Models
• Automated Debugging
• Prolog
• Spectrum-based Debugging
• Mutation-based Repair