ADEPT: A Unified Framework for Deep Learning Test Adequacy

2026-08-12

AI Summary

Existing approaches to measuring test adequacy in deep learning are fragmented, require complex configuration, and suffer from poor reproducibility, hindering both research and practical adoption. To address these limitations, this work proposes ADEPT, a unified framework that integrates diverse test adequacy metrics—including neuron coverage, boundary coverage, and mutation scores—into a single platform for the first time. ADEPT employs YAML-based configuration management, reusable preprocessing caches, and templated metric interfaces to ensure a consistent execution pipeline and support extensible architecture. This design substantially lowers the barrier to entry, enhances experimental reproducibility, and improves comparability of results, enabling researchers and practitioners to efficiently incorporate comprehensive test adequacy evaluation into their development workflows.

Abstract

Over the past decade, many test adequacy metrics have been proposed for deep learning that characterize test dataset adequacy from different perspectives, e.g., neuron activation behavior, latent feature coverage, decision-boundary exploration, etc. However, these metrics are typically released as independent research prototypes with substantially different installation and preprocessing requirements, execution workflows, and configuration mechanisms. These complications make them quite difficult to reproduce, compare, and adopt in research work and practical deployment alike. In this paper, we present the engineering details of ADEPT, a framework that integrates representative adequacy techniques, including neuron-coverage-based metrics, surprise adequacy, input distribution coverage, boundary coverage, and source- and model-level mutation score, under a consistent execution workflow. ADEPT provides a template-based metric interface with well-defined extension points for integrating new adequacy metrics. Furthermore, it provides YAML-based configuration management, preprocessing-cache reuse, and structured result reporting, making it easy to use in any research and development workflows. ADEPT is designed for researchers and practitioners who wish to reproduce and apply adequacy metrics without spending days or weeks implementing missing tooling or configuring disparate research prototypes. A demo video is available at https://aub.ie/ADEPT_video.

Problem

• test adequacy
• deep learning
• reproducibility
• tool integration
• metric comparison

Innovation

• test adequacy
• deep learning testing
• unified framework
• metric integration
• reproducibility