Test Amplification for REST APIs via Single and Multi-Agent LLM Systems

2025-04-10

AI Summary

Low test coverage and difficulty in detecting boundary defects hinder REST API testing. Method: This paper proposes an automated test augmentation framework leveraging both single-agent and multi-agent large language models (LLMs), integrating API contract analysis, collaborative reasoning, and test generation-execution to systematically extend existing test suites and deeply explore boundary scenarios—including HTTP methods, paths, status codes, and anomalous inputs. Contribution/Results: We present the first comparative evaluation of these two LLM agent paradigms: multi-agent LLMs significantly outperform single-agent ones in path and status-code coverage (+28.6% on average) and real-world defect detection (uncovering seven previously unreported bugs); conversely, single-agent LLMs achieve higher computational efficiency and lower energy consumption (−39.2%). The study quantitatively characterizes the trade-offs among coverage, defect detection, and energy efficiency, establishing a reproducible methodology and empirical benchmark for LLM-driven API testing.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Multiagent Systems: Coordination and Collaboration

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

REST APIs (Representational State Transfer Application Programming Interfaces) are essential to modern cloud-native applications. Strong and automated test cases are crucial to expose lurking bugs in the API. However, creating automated tests for REST APIs is difficult, and it requires test cases that explore the protocol's boundary conditions. In this paper, we investigate how single-agent and multi-agent LLM (Large Language Model) systems can amplify a REST API test suite. Our evaluation demonstrates increased API coverage, identification of numerous bugs in the API under test, and insights into the computational cost and energy consumption of both approaches.

Problem

• Amplifying REST API test suites using LLM systems
• Improving API coverage and bug detection via agents
• Comparing computational costs of single vs multi-agent approaches

Innovation

• Single-agent LLM for REST API test amplification
• Multi-agent LLM system to enhance API coverage
• Analyzing computational cost and energy consumption