Ising-based Test Optimization and Benchmarking

2026-04-12

AI Summary

This work addresses the combinatorial optimization challenge of test case selection and minimization in software testing by proposing IsingTester, a novel framework that introduces Coherent Ising Machines (CIMs) to the domain of test optimization for the first time. The approach formulates the problem as an Ising model, encoding optimization objectives into spin configurations, and integrates multiple solvers—including CIM simulation and exhaustive search—into an end-to-end automated optimization pipeline. To facilitate systematic evaluation and comparison of diverse solution strategies, the authors also release IsingBench, a benchmark platform featuring extensible modeling interfaces and a reproducible experimental environment.

Technology Category

• Constraint Satisfaction and Optimization: Solvers and Tools
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

Test optimization contains test case selection and minimization, which is an important challenge in software testing and has been addressed with search-based approaches intensively in the past. Inspired by the recent advancement of using quantum optimization solutions for addressing test optimization problems, we looked into Coherent Ising Machines (CIM), which offer potential for solving combinatorial optimization problems, but have not yet been exploited in test optimization. Hence, in this paper, we present IsingTester, an open-source, Python-based command-line tool that provides an end-to-end pipeline for solving test optimization problems that are formulated as Ising models. With IsingTester, we reformulate test selection and minimization as Ising spin configurations, encode multiple optimization strategies into Ising Hamiltonians, and implement solvers including CIM simulation and brute-force search. Given a user-provided dataset and solver configuration, IsingTester automatically performs problem encoding, optimization, and spin decoding, returning selected test cases back to the user. Along with IsingTester, we also present the accompanying IsingBench for evaluating and comparing optimization techniques across Ising-based paradigms against baseline approaches. A screencast demonstrating the tool is available at: https://github.com/WSE-Lab/IsingBench.

Problem

• test optimization
• test case selection
• test minimization
• combinatorial optimization
• software testing

Innovation

• Ising model
• Coherent Ising Machine
• test optimization
• combinatorial optimization
• software testing