Autopoiesis: A Self-Evolving System Paradigm for LLM Serving Under Runtime Dynamics

2026-04-08

AI Summary

This work addresses the limitations of traditional large language model (LLM) serving systems, which rely on static, hand-crafted scheduling policies that struggle to adapt to runtime dynamics such as load fluctuations and cluster elasticity. To overcome this, the authors propose an online self-evolving system that leverages an LLM-driven program synthesis pipeline to continuously observe system states and autonomously rewrite serving policy code in real time. By transforming serving policies from fixed artifacts into “living code” continuously optimized by an LLM during deployment, this approach establishes a new paradigm for self-evolving LLM serving systems. Experimental results demonstrate that, across diverse dynamic scenarios, the proposed system achieves an average performance improvement of 34% over state-of-the-art baselines, with gains reaching up to 53%.

Abstract

Modern Large Language Model (LLM) serving operates in highly volatile environments characterized by severe runtime dynamics, such as workload fluctuations and elastic cluster autoscaling. Traditional serving systems rely on static, human-engineered serving policies (e.g., scheduling algorithms and rescheduling strategies) to manage these dynamics. However, these policies must navigate deeply intertwined runtime trade-offs (e.g., scheduling overhead vs. execution efficiency, rescheduling frequency vs. reconfiguration overhead), whose optimal balance is workload-specific and shifts continuously as runtime conditions evolve, rendering any fixed policy fundamentally unable to adapt. We propose Autopoiesis, a novel online self-evolving system that shifts LLM serving from static policy deployment to continuous online policy evolution. First, Autopoiesis introduces an LLM-driven program synthesis workflow to evolve serving policies with respect to real-time observed dynamics, where the evolved policies reflect the optimal decision in navigating the complex, multi-dimensional trade-off space. Second, Autopoiesis enables this synthesis process to operate continuously during serving, observing real-world system behavior, and rewriting the policy code as runtime trade-offs shift, thereby transforming policy design from a one-time offline endeavor into an ongoing system component, enabling autonomous adaptation to evolving runtime conditions. Together, we establish a new paradigm: Serving policies are no longer static artifacts designed by humans before deployment, but living code that LLMs continuously evolve throughout deployment to navigate runtime trade-offs beyond human design. We evaluate Autopoiesis across diverse runtime dynamics and show up to 53% and on average 34% improvements over state-of-the-art LLM serving systems.

Problem

• LLM serving
• runtime dynamics
• serving policies
• trade-offs
• autonomous adaptation

Innovation

• self-evolving system
• LLM-driven program synthesis
• runtime dynamics
• online policy evolution
• autonomous adaptation