Prompt-Based REST API Test Amplification in Industry: An Experience Report

2026-01-25

AI Summary

This study addresses the lack of empirical evaluation of large language models (LLMs) in industrial-scale REST API testing, particularly regarding their effectiveness under complex authentication schemes, stateful interactions, and organizational constraints. For the first time, we conduct a systematic validation within the live production environment of a major Belgian logistics enterprise, targeting six representative microservice endpoints. Leveraging a prompt-engineering-based approach to LLM-driven test case augmentation integrated with an industrial-grade microservice testing framework, our experiments significantly improve test coverage and uncover multiple system anomalies and observability gaps. This work provides the first empirical evidence of the practical utility—and inherent limitations—of LLM-augmented testing in security-sensitive, state-intensive industrial settings.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Security and Privacy: Large-scale security measurements

Abstract

Large Language Models (LLMs) are increasingly used to support software testing tasks, yet there is little evidence of their effectiveness for REST API testing in industrial settings. To address this gap, we replicate our earlier work on LLM-based REST API test amplification within an industrial context at one of the largest logistics companies in Belgium. We apply LLM-based test amplification to six representative endpoints of a production microservice embedded in a large-scale, security-sensitive system, where there is in-depth complexity in authentication, stateful behavior, and organizational constraints. Our experience shows that LLM-based test amplification remains practically useful in industry by increasing coverage and revealing various observations and anomalies.

Problem

• REST API testing
• Large Language Models
• industrial setting
• test amplification
• software testing

Innovation

• LLM-based test amplification
• Prompt-based testing
• REST API testing
• Industrial evaluation
• Test coverage