Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies

2026-06-22

AI Summary

This work addresses novel security and privacy threats introduced by self-evolving large language model (LLM) agents, whose autonomous updating capabilities enable attacks to become permanently encoded, amplified across generations, and propagated within agent populations. The paper proposes a Module–Lifecycle Attack Surface (MLAS) analysis framework that systematically characterizes 25 distinct attack surfaces arising from the intersection of five functional modules and five lifecycle stages, revealing seven previously undocumented cross-module synergistic amplification effects. Findings demonstrate that self-evolution transforms all known attacks from session-bound to lineage-persistent and introduces entirely new attack classes. Evolution-native designs increase the attack surface by 3.5×, achieve 100% attack persistence, and render existing colocated scanners effective against only 2.5% of threats. Empirical evaluation identifies 17 high-risk attack surfaces lacking adequate mitigations.

Abstract

Self-evolving LLM agent systems, which autonomously update their model parameters, memory, tools, and architectures, introduce a qualitatively new threat landscape in which adversarial influences become permanently encoded, self-amplify across generations, and propagate through populations without sustained attacker access. We present a systematic security and privacy analysis organized around the Module-Lifecycle Attack Surface (MLAS) matrix, which decomposes the attack surface into five functional modules (Brain, Cognitive Resource, Execution, Self-Design, Collective) $\times$ five lifecycle stages (Bootstrap, Propose, Evaluate, Commit, Serve). Analysis of the resulting 25 cells reveals that 17 face critical threats for which no effective partial mitigation. We identify seven cross-cutting amplification effects that interact synergistically and cannot be addressed by securing individual modules in isolation. Comparative case studies of two open-source frameworks demonstrate that evolution-native design activates $3.5\times$ more attack surface cells and achieves a 100% attack persistence rate (40/40 payloads across all CIA+Privacy categories), while co-located security scanners block only 2.5% of attacks. Our findings establish that self-evolution converts every known attack category from session-bounded to lineage-persistent, gives rise to entirely new attack classes, and renders static defenses structurally inadequate, motivating evolution-aware security frameworks and formal verification for self-modifying systems.

Problem

• self-evolving LLM agents
• security threats
• attack persistence
• amplification effects
• evolution-aware security

Innovation

• self-evolving LLM agents
• Module-Lifecycle Attack Surface (MLAS)
• attack amplification
• lineage-persistent threats
• evolution-aware security