WebCQ: Cooperative Multi-Agent Deep Reinforcement Learning for Scalable Web GUI Testing

2026-06-21

AI Summary

This work addresses the limited scalability of existing multi-agent reinforcement learning approaches for GUI testing, which rely on tabular methods ill-suited for large-scale web applications. To overcome this challenge, the authors propose WebCQ, the first framework integrating Deep Q-Networks (DQN) with QTRAN, enhanced by dynamic action space modeling that fuses UI semantics and exploration features, alongside a lightweight asynchronous synchronization mechanism. These innovations significantly improve collaborative exploration and scalability in complex web environments. Experimental evaluation on eight large commercial websites demonstrates that, under identical resource constraints, WebCQ explores 33.3% more states and executes 42.2% more unique actions than MARG, uncovering more faults. Furthermore, during 20-hour stress tests, WebCQ exhibits superior throughput and scales effectively with increasing numbers of agents.

Abstract

Multi-agent reinforcement learning (MARL)-based techniques have shown promise for GUI testing. However, as the complexity of modern GUI software increases, existing MARL-based approaches (e.g., MARG and Fastbot) struggle to scale due to the inherent limitations of their underlying tabular reinforcement learning algorithms. This limits their applicability to large-scale commercial GUI software, especially web applications with vast state spaces and many interactive elements. To fill this gap, we propose WebCQ, a novel MARL-based approach for scalable web GUI testing. WebCQ incorporates QTRAN for multi-agent coordination and a lightweight synchronization mechanism, allowing it to work under asynchronous web testing scenarios. It extracts semantic and exploration features for each UI event to form an action vector. This vector is concatenated with the current state vector and fed into the policy network, enabling DQN-based decision making within a dynamic action space. We evaluated WebCQ on eight large-scale commercial websites. Under the same time budget and agent count, WebCQ explored 33.3% more states and executed 42.2% more unique actions than MARG, while triggering more failures on six of the eight websites under test. It also demonstrated strong scalability, maintaining higher action throughput during 20-hour experiments, and achieving greater performance improvements as the number of agents increased. These results show that WebCQovercomes key limitations of existing MARL-based approaches, providing a scalable and effective solution for enhancing modern web GUI testing.

Problem

• scalable web GUI testing
• multi-agent reinforcement learning
• large state space
• interactive elements
• commercial web applications

Innovation

• multi-agent reinforcement learning
• QTRAN
• dynamic action space
• web GUI testing
• scalable testing