complex systems

Existing chaos engineering tools rely heavily on manual experiment design and configuration tuning, resulting in low efficiency and high operational costs. Method: This paper proposes the first automated chaos engineering system that deeply integrates large language models (LLMs) into a full闭环—spanning requirement understanding, experiment design, infrastructure-as-code (IaC)-based code generation, automated debugging, and validation-driven repair—all orchestrated by the LLM. Contribution/Results: The approach achieves end-to-end automation from requirement specification to effectiveness verification, eliminating manual intervention and significantly reducing testing time and operational overhead. Evaluated across multi-scale systems, it successfully executes single, semantically coherent chaos experiments with validated effectiveness meeting industrial usability standards. This work establishes a reusable technical paradigm for LLM-augmented reliability engineering.

Current chaos engineering (CE) relies heavily on manual experiment design and result analysis, incurring high labor costs and steep expertise barriers. This paper proposes ChaosEater—the first automated framework systematically integrating large language model (LLM)-based agent workflows across the full CE lifecycle: elasticity requirement interpretation, experiment plan generation, Kubernetes-native fault injection, automated resilience testing, and actionable recovery recommendations. Its key contributions are: (1) the first end-to-end LLM-driven CE闭环—from requirement specification to repair strategy—fully eliminating the need for domain experts; and (2) a novel architecture combining hierarchical task decomposition, domain-knowledge augmentation, and multi-source validation, empirically validated across heterogeneous Kubernetes clusters of varying scales. Experimental results demonstrate significant reductions in both human effort and operational cost. Generated experiment plans and remediation suggestions are rigorously evaluated by both human engineers and independent LLMs, confirming their practicality, correctness, and domain alignment.

Chaos engineering lacks a systematic, comprehensive review in the literature. Method: This paper conducts the first multi-source literature review (MLR), systematically analyzing 96 academic and gray literature sources published between 2016 and 2024—including 88 core publications from 2019 to 2024. It synthesizes findings via thematic clustering and qualitative coding. Contribution/Results: The study establishes the first consensus definition of chaos engineering, proposes a four-layer capability model and a five-dimensional component taxonomy, and performs a cross-tool evaluation of 12 mainstream chaos engineering tools. It identifies six open research challenges and clarifies practice drivers, tool characteristics, and research evolution trends. The results provide a foundational theoretical framework, methodological benchmark, and roadmap for future work—bridging critical knowledge gaps between academia and industry.

Microservice systems are prone to cascading failures due to strong inter-service dependencies, and conventional chaos engineering relies on costly fault injection in production environments. This paper proposes a lightweight cascading failure prediction method: it automatically constructs a high-fidelity service dependency graph from distributed tracing data and performs Monte Carlo–based stochastic fault propagation simulations on this graph to enable rapid resilience assessment at the design stage—without requiring real-world fault injection. We provide the first theoretical proof that the automatically derived dependency graph supports high-accuracy resilience prediction. Evaluation on a Social Network benchmark shows prediction errors ≤ 0.0004 against ground-truth measurements; mean absolute error (MAE) is 0.025 under no-replica configurations and exactly zero with replicas, demonstrating highly accurate availability estimation capability.

This work addresses the limitations of existing hardware fault injection tools, which often lack efficiency and flexibility for systematically evaluating the reliability and fault tolerance of computing systems. To overcome these challenges, the authors present the first modular, open-source, and highly configurable fault injection framework integrated into the gem5 simulator. The framework enables precise injection of both hardware and software faults across multiple architectural levels—from registers to caches—and supports sophisticated fault models coupled with fine-grained triggering mechanisms. By offering unprecedented control and scalability, this infrastructure significantly enhances the ability to assess fault-tolerant mechanisms and resilience strategies, thereby providing a powerful and flexible experimental platform for advancing research in high-reliability, high-performance computing systems.

Distributed systems operating in complex environments are highly susceptible to failures and adversarial behaviors, making their performance difficult to predict directly from formal designs. This work proposes the first framework that systematically enables performance prediction from formal models by integrating a reusable fault-injection library with a modeling methodology based on the Maude language. The authors develop an automated tool, PERF, which combines model composition and statistical analysis techniques to accurately estimate key performance metrics—such as throughput and latency—across diverse failure scenarios. Experimental evaluation demonstrates that PERF’s predictions align closely with measurements from real-world deployments on representative distributed systems, significantly enhancing the practical utility of formal methods in performance assessment.

Traditional distributed systems struggle to support modern autonomous infrastructures that integrate stochastic models and autonomous agents. This work proposes the Post-Deterministic Distributed System (PDDS) model, introducing for the first time its five architectural pillars. Its core innovation is a "cognitive state replication" mechanism that extends consistency from data visibility to knowledge visibility, alongside a novel fault classification framework. By leveraging protocol-driven development, verifiable agent infrastructure, and semantic quorum guarantees, PDDS enables coordination among semantically equivalent yet executionally divergent agents. This approach achieves verifiable semantic rollback and cross-agent reasoning consistency, establishing a theoretical foundation for trustworthy autonomous systems.

This work addresses the challenge of ensuring safety, reliability, and trustworthiness in collective adaptive systems operating in dynamic environments by proposing a modular design paradigm centered on intrinsic trustworthiness. The approach integrates a runtime model based on local causal event sequences, a temporal logic verification technique supporting modular architectures, and a compositional reasoning mechanism for global system properties grounded in component attributes. Through this tripartite framework, the study overcomes key limitations of conventional formal methods and demonstrates substantial improvements in verifiability and scalability in case studies, thereby establishing both a theoretical foundation and a practical pathway for engineering highly trustworthy collective adaptive systems.

While microservice failures are readily detectable, root cause analysis remains inefficient due to alarm flooding and the absence of structured memory capturing system dependencies and historical behaviors. This work proposes a topology-aware, operation-memory-driven multi-agent architecture that decouples root cause inference from explanation for the first time: the former relies on deterministic computation using a learned dependency graph and temporal anomaly thresholds, while the latter leverages a large language model to generate interpretable recommendations grounded in structured evidence. A novel four-layer operational memory mechanism enables traceable and reusable autonomous operations. Evaluated on an e-commerce benchmark platform with eight types of injected faults, the approach successfully reproduces and resolves two real-world cascading failures, significantly improving diagnostic accuracy and efficiency.

Existing SRE benchmark tasks are overly simplified and fail to capture the complexity of fault diagnosis and mitigation in real-world production environments. This work proposes the first high-fidelity, scalable evaluation benchmark for SRE agents, built upon a realistic cloud-native system stack that dynamically simulates operational conditions. The benchmark incorporates a fault injector and noise simulator to support diverse failure modes—including metastable and correlated failures—and provides 90 realistic, challenging tasks. Designed with a modular architecture, it enables continuous extension and adaptation. Experimental results demonstrate significant performance disparities among state-of-the-art AI agents across different fault types, with end-to-end success rates varying by up to 40%, thereby validating the benchmark’s effectiveness and inherent difficulty.