AutoRestTest at the SBFT 2026 Tool Competition

2026-07-01

AI Summary

This work addresses the inefficiency in black-box REST API testing caused by the vast input space and complex operation dependencies. The authors propose a novel approach that integrates semantic attribute dependency graphs, multi-agent reinforcement learning, and large language models. By uniquely combining semantic dependency modeling with intelligent exploration mechanisms, the method enables automatic, efficient, and precise testing of high-dimensional API interaction spaces. Evaluated on the SBFT 2026 REST League benchmark, the approach achieves state-of-the-art performance: within one hour, it discovers an average of 67.09 unique server-side errors per API and successfully executes 17.27 operations across 11 APIs (comprising 317 operations in total), ranking first in fault detection capability, testing efficiency, and effectiveness.

Abstract

Large input spaces and complex inter-operation dependencies make black-box REST API testing challenging. AutoRestTest combines a Semantic Property Dependency Graph, multi-agent reinforcement learning, and large language models to intelligently explore large API input spaces. In the SBFT 2026 REST League, AutoRestTest ranked first in all three evaluation categories -- fault detection, overall efficiency, and overall effectiveness -- on 11 APIs (317 operations, approximately 29 per API), averaging 67.09 unique server errors and 17.27 successfully processed operations per API under a one-hour testing budget.

Problem

• black-box testing
• REST API
• input space
• inter-operation dependencies
• fault detection

Innovation

• Semantic Property Dependency Graph
• multi-agent reinforcement learning
• large language models
• black-box REST API testing
• input space exploration