🤖 AI Summary
Existing chaos engineering tools rely heavily on manual experiment design and configuration tuning, resulting in low efficiency and high operational costs.
Method: This paper proposes the first automated chaos engineering system that deeply integrates large language models (LLMs) into a full闭环—spanning requirement understanding, experiment design, infrastructure-as-code (IaC)-based code generation, automated debugging, and validation-driven repair—all orchestrated by the LLM.
Contribution/Results: The approach achieves end-to-end automation from requirement specification to effectiveness verification, eliminating manual intervention and significantly reducing testing time and operational overhead. Evaluated across multi-scale systems, it successfully executes single, semantically coherent chaos experiments with validated effectiveness meeting industrial usability standards. This work establishes a reusable technical paradigm for LLM-augmented reliability engineering.
📝 Abstract
Chaos Engineering (CE) is an engineering technique aimed at improving the resiliency of distributed systems. It involves artificially injecting specific failures into a distributed system and observing its behavior in response. Based on the observation, the system can be proactively improved to handle those failures. Recent CE tools realize the automated execution of predefined CE experiments. However, defining these experiments and reconfiguring the system after the experiments still remain manual. To reduce the costs of the manual operations, we propose extsc{ChaosEater}, a extit{system} for automating the entire CE operations with Large Language Models (LLMs). It pre-defines the general flow according to the systematic CE cycle and assigns subdivided operations within the flow to LLMs. We assume systems based on Infrastructure as Code (IaC), wherein the system configurations and artificial failures are managed through code. Hence, the LLMs' operations in our extit{system} correspond to software engineering tasks, including requirement definition, code generation and debugging, and testing. We validate our extit{system} through case studies on both small and large systems. The results demonstrate that our extit{system} significantly reduces both time and monetary costs while completing reasonable single CE cycles.