Human-AI Collaboration for Scaling Agile Regression Testing: An Agentic-AI Teammate from Manual to Automated Testing

2026-03-09

AI Summary

This work addresses the bottleneck in agile development where the rapid generation of test specifications far outpaces their manual conversion into executable scripts, leading to backlogs and delayed releases. To bridge this gap, the authors propose an embodied AI collaborator architecture tailored for industrial settings, featuring a retrieval-augmented multi-agent system that enables end-to-end generation of system-level automated test scripts from validated specifications. The framework seamlessly integrates into existing agile workflows while preserving human oversight through mandatory review gates. Evaluated in a real-world deployment at Hacon, a Siemens subsidiary, the approach significantly reduces the manual scripting burden and demonstrates that clear specification standards combined with effective human-AI collaboration are pivotal for scaling regression test automation.

Technology Category

• Intelligent Robots: Embodied AI
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Agent/AI Theories and Architectures

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Agile organizations increasingly rely on automated regression testing to sustain rapid, high-quality software delivery. However, as systems grow and requirements evolve, a persistent bottleneck arises: test specifications are produced faster than they can be transformed into executable scripts, leading to mounting manual effort and delayed releases. In partnership with Hacon (a Siemens company), we present an agentic AI approach that generates system-level test scripts directly from validated specifications, aiming to accelerate automation without sacrificing human oversight. Our solution features a retrieval-augmented, multi-agent architecture integrated into Hacon's agile workflows. We evaluate this system through a mixed-method analysis of industrial artifacts and practitioner feedback. Results show that the AI teammate significantly increases test script throughput and reduces manual authoring effort, while underscoring the ongoing need for clear specifications and human review to ensure quality and maintainability. We conclude with practical lessons for scaling regression automation and fostering effective Human-AI collaboration in agile environments.

Problem

• regression testing
• test automation
• agile development
• manual-to-automated transition
• testing bottleneck

Innovation

• agentic AI
• retrieval-augmented generation
• automated regression testing
• human-AI collaboration
• agile software development