AIChilles: Automatically Uncovering Hidden Weaknesses in AI-Evolved Systems

2026-06-14

AI Summary

AI evolution systems are prone to latent defects in correctness, performance, or scalability under unseen workloads, necessitating automated detection mechanisms. This work proposes the first automated vulnerability discovery framework tailored for AI evolution systems, integrating deterministic workload parameter extraction, agent-driven constraint inference, differential oracles, and code-frequency coverage. Through a multidimensional feedback mechanism, the framework efficiently identifies regression issues across diverse workloads. Evaluated on five systems and thirty AI-evolved programs, it uncovered 49 distinct classes of hidden weaknesses, demonstrating its effectiveness and compatibility with AI-driven development pipelines for proactive defect prevention.

Abstract

The computer systems community has recently seen growing interest in AI-driven system evolution, where AI agents iteratively rewrite systems. Frameworks such as AdaEvolve and Engram report 12-60% score improvements over human-designed algorithms. While these results are promising, there are practical concerns if these AI-evolved programs can perform worse on unseen workloads and exhibit scalability regressions. Given the speed and scale of AI-generated code, we need automated mechanisms to uncover such identify hidden weaknesses in AI-evolved systems programs. To this end, we develop AIChilles that takes as input a baseline program $P$ and an AI-evolved program $P'$, AIChilles searches for valid workloads where $P'$ regresses relative to $P$ in correctness, runtime, memory usage, or output quality. To tackle the diversity in system applications, weakness types and potential bugs, AIChilles combines deterministic workload-parameter extraction, agent-based constraint inference, differential oracles, and code-frequency coverage to discover diverse failures. Across five system applications and 30 AI-evolved programs, AIChilles finds 49 distinct hidden weaknesses. We also show that explicitly including AIChilles in the AI-driven development lifecycle can mitigate several of these weaknesses.

Problem

• AI-evolved systems
• hidden weaknesses
• scalability regression
• unseen workloads
• performance regression

Innovation

• AI-evolved systems
• automated weakness detection
• differential oracles
• constraint inference
• code coverage