Delta Debugging in the Absence of Test Oracles Through Metamorphic Testing

2026-07-01

AI Summary

This work addresses the limitation of traditional delta debugging, which relies on test oracles to verify whether reduced inputs preserve the target property—a requirement that hinders its applicability in oracle-absent scenarios. To overcome this challenge, the paper introduces metamorphic testing into the delta debugging pipeline for the first time, constructing an oracle-free validation function that replaces the original oracle-dependent testing mechanism. The proposed approach, termed DDMT, seamlessly integrates with various delta debugging algorithms. Empirical evaluation across 66 subjects demonstrates that DDMT not only maintains or even improves reduction effectiveness and query efficiency but also significantly extends the applicability of delta debugging to environments lacking test oracles.

Abstract

Delta debugging provides an automatic way to minimize a program input while preserving a certain property. However, its effectiveness fundamentally relies on the availability of test oracles to determine whether a reduced input still preserves the specific property. Consequently, the oracle problem substantially limits the applicability of existing delta debugging techniques, particularly for oracle-deficient programs where output correctness cannot be directly determined. To address this problem, this paper proposes a novel approach, DDMT, to enhance the applicability of delta debugging, especially facilitating its application to oracle-deficient programs. Our key insight is to redesign an oracle-independent test function and incorporate it into the reduction procedure of delta debugging such that the property-preservation validation can be accomplished without requiring a test oracle. To this end, DDMT employs the technique of metamorphic testing, which is a property-based and oracle-independent testing method. It establishes a metamorphic testing-based test function, using it as a replacement for the original test function adopted by delta debugging. The experiments evaluate DDMT on 66 subjects across both oracle-available and oracle-deficient scenarios, with different delta debugging approaches. The results positively confirm that DDMT can enhance the applicability of delta debugging while often preserving or improving reduction effectiveness and query efficiency. Furthermore, compared to the relevant delta debugging approaches, DDMT is also able to achieve performance improvements with proper configurations.

Problem

• delta debugging
• test oracle
• oracle problem
• metamorphic testing
• oracle-deficient programs

Innovation

• Delta Debugging
• Metamorphic Testing
• Test Oracle
• Oracle-deficient Programs
• Input Minimization