WebMAC: A Multi-Agent Collaborative Framework for Scenario Testing of Web Systems

2026-04-15

AI Summary

Existing large language model–based approaches to web scenario testing struggle to effectively uncover defects due to incomplete natural language descriptions and the neglect of test adequacy criteria. To address this limitation, this work proposes WebMAC, a novel multi-agent collaborative framework that introduces multi-agent coordination into web testing for the first time. WebMAC interactively clarifies ambiguous user intents to enrich scenario semantics and leverages equivalence class partitioning to generate instantiated test scenarios that satisfy established adequacy criteria, which are then automatically translated into executable test scripts. Empirical evaluation on four real-world web systems demonstrates that, compared to state-of-the-art methods, WebMAC improves test script execution success rates by 30%–60%, enhances testing efficiency by 29%, reduces token consumption by 47.6%, and uncovers a greater number of system faults.

Technology Category

• Multiagent Systems: Coordination and Collaboration
• Data Mining & Knowledge Management: Knowledge Acquisition from the Web
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Scenario testing is an important technique for detecting errors in web systems. Testers draft test scenarios and convert them into test scripts for execution. Early methods relied on testers to convert test scenarios into test scripts. Recent LLM-based scenario testing methods can generate test scripts from natural language descriptions of test scenarios. However, these methods are not only limited by the incompleteness of descriptions but also overlook test adequacy criteria, making it difficult to detect potential errors. To address these limitations, this paper proposes WebMAC, a multi-agent collaborative framework for scenario testing of web systems. WebMAC can complete natural language descriptions of test scenarios through interactive clarification and transform adequate instantiated test scenarios via equivalence class partitioning. WebMAC consists of three multi-agent modules, responsible respectively for completing natural language descriptions of test scenarios, transforming test scenarios, and converting test scripts. We evaluated WebMAC on four web systems. Compared with the SOTA method, WebMAC improves the execution success rate of generated test scripts by 30%-60%, increases testing efficiency by 29%, and reduces token consumption by 47.6%. Furthermore, WebMAC can effectively detect more errors in web systems.

Problem

• scenario testing
• web systems
• test adequacy
• natural language description
• error detection

Innovation

• multi-agent collaboration
• scenario testing
• equivalence class partitioning
• test script generation
• web system testing