Evolution-Aware Regression Test Prioritization of ML-Enabled Systems Using Gradient-Based Behavior Vectors

2026-06-26

AI Summary

This work addresses the challenge that evolving machine learning systems can render existing test cases ineffective, necessitating efficient identification of performance-degrading tests for regression testing prioritization. The authors propose GBV-PD, a novel approach that introduces low-dimensional behavioral vectors into evolution-aware regression testing. By constructing behavior vectors from the loss gradients of the original model and incorporating the direction of model parameter updates, GBV-PD predicts the trend of loss changes for test cases on the updated model—without requiring actual execution of the new model—enabling highly efficient test prioritization. The method features a directional prediction mechanism and supports caching of behavioral vectors to handle multiple model updates. Experiments demonstrate that GBV-PD significantly outperforms non-directional baselines on both classification and regression tasks, achieving performance close to full-gradient reference methods while substantially reducing computational time and storage overhead.

Abstract

The machine learning(ML) component of an ML-enabled system evolves through retraining, fine-tuning, and optimization, so previously valid test results may no longer hold. A single evolution step can worsen performance on some test cases while improving others, making regression test prioritization inherently directional. We present Gradient-based Behavior Vector-Parameter Delta(GBV-PD), the first approach to operationalize the behavior vector space for evolution-aware regression test prioritization. GBV-PD represents each test case as a gradient-based vector(GBV), a low-dimensional projection of its loss gradient under the original model. It then projects the observed parameter update of the evolved model onto the same PCA basis and uses the resulting alignment to estimate whether each test case's loss is likely to increase or decrease, without running the evolved model on test cases during prioritization. In an empirical study across classification and regression tasks, GBV-PD consistently outperformed non-directional baselines and remained competitive with a full-gradient reference, while offering better time and storage profiles for repeated updates via reusable GBV caching. These results show that behavior-space ideas can be operationalized into a practical and efficient mechanism for repeated-update regression testing of evolving ML-enabled systems.

Problem

• regression testing
• test prioritization
• ML-enabled systems
• model evolution
• behavior vector

Innovation

• gradient-based behavior vector
• evolution-aware testing
• regression test prioritization
• ML-enabled systems
• parameter delta projection