state management

Cloud application development has long suffered from high expertise barriers due to the need to integrate distributed systems, database, and software engineering knowledge. This paper proposes Stateflow—the first cloud-native, streaming, stateful function-computing framework—designed to jointly address programmability, strongly consistent fault-tolerant transactions, and serverless semantics. Its three key contributions are: (1) an object-oriented declarative programming model that eliminates explicit error handling; (2) the Styx engine, which guarantees deterministic multi-partition serializability, snapshot consistency, and zero-loss state migration; and (3) a streaming dataflow execution model with transaction-aware state migration, enabling dynamic elastic scaling. Experimental evaluation demonstrates that Stateflow significantly outperforms existing systems in both throughput and recovery performance, substantially reducing development complexity for high-concurrency, strongly consistent cloud applications.

This work addresses the inefficiencies in existing checkpointing schemes for large-scale Transformer training, which overlook the three-dimensional heterogeneity of state data—spanning memory placement, logical sharding, and serialization requirements—leading to device-host transfer bottlenecks, inefficient serialization, and I/O contention. To resolve this, the authors propose DataStates-LLM, a novel architecture that introduces, for the first time, a composable state provider mechanism to explicitly model state heterogeneity and decouple state abstraction from data movement. By leveraging parameter invariance during forward and backward passes, it enables non-blocking asynchronous snapshots and co-optimizes metadata serialization with tensor I/O. Evaluated on a 256×A100-40GB system training a 70B-parameter model, the approach achieves up to a 4× improvement in checkpoint throughput and reduces end-to-end training time by as much as 2.2×.

Kubernetes lacks native support for live migration of stateful microservices, hindering elastic scheduling of stateful workloads. Method: This paper proposes MS2M, a framework integrating message-driven migration with forensics-grade container checkpointing to enable efficient, low-downtime in-memory state migration for StatefulSets. It introduces a novel dynamic threshold truncation mechanism based on message rate, combined with incremental checkpointing and optimized state propagation. Contribution/Results: Experiments demonstrate a 96.986% reduction in per-Pod migration downtime. StatefulSet-granularity migration preserves service continuity while enhancing orchestration flexibility. MS2M provides a practical, deployable solution for elastic scheduling of cloud-native stateful workloads, marking the first approach to achieve efficient, consistent live migration of StatefulSet memory state.

This work addresses the challenge of resource dynamics in shared clusters during large language model training, where conventional checkpoint-based state migration incurs substantial I/O overhead and GPU stalls. To overcome this, the authors propose ETC, a novel framework that eliminates checkpointing entirely by exploiting state locality and leveraging peer-to-peer GPU communication with communication aggregation to enable storage-free, low-fragmentation elastic state migration. Seamlessly integrated into the Megatron-LM hybrid parallel training system, ETC reduces migration overhead by 2.33–6.37× compared to checkpoint-based approaches across diverse parallelism configurations, significantly enhancing the practicality and efficiency of elastic training for large models.

This work addresses the high latency in real-time stream processing caused by tight coupling between state I/O and the data path, which blocks the CPU on the critical path. To mitigate this, the authors propose Keyed Prefetching, a mechanism that extracts state-access keys from upstream operators and proactively prefetches the required state, thereby overlapping I/O with computation to hide latency. Complementing this, a Timestamp-Aware Caching strategy is introduced to efficiently manage prefetched and historical states in memory. Together, these techniques significantly reduce end-to-end latency for long-running real-time queries while maintaining high throughput and effectively decoupling state access from data processing.

Existing large model training data pipelines struggle to simultaneously ensure batch semantics, fault isolation, and consistency. This work proposes an agent-free, object-storage-native training data plane featuring three core innovations: a Transactional Global Batch (TGB) abstraction that guarantees training consistency, a storage-layer-embedded garbage collection mechanism aligning producer states with distributed checkpoints, and a communication-free Decentralized Adaptive Commit (DAC) algorithm. Leveraging lakehouse ACID semantics in object storage and distributed checkpointing, the system achieves higher throughput than colocated loaders and Kafka, lower read latency, and full fault isolation across 64-GPU multimodal pretraining and supervised fine-tuning (SFT) workloads.

This work addresses the limitations of traditional large language model (LLM) serving systems, which rely on static, hand-crafted scheduling policies that struggle to adapt to runtime dynamics such as load fluctuations and cluster elasticity. To overcome this, the authors propose an online self-evolving system that leverages an LLM-driven program synthesis pipeline to continuously observe system states and autonomously rewrite serving policy code in real time. By transforming serving policies from fixed artifacts into “living code” continuously optimized by an LLM during deployment, this approach establishes a new paradigm for self-evolving LLM serving systems. Experimental results demonstrate that, across diverse dynamic scenarios, the proposed system achieves an average performance improvement of 34% over state-of-the-art baselines, with gains reaching up to 53%.

This work addresses the high latency, resource contention, and operational overhead caused by frequent state updates in streaming machine learning. The authors propose a probabilistic sparsification strategy that decouples inference from state persistence: while all events contribute to inference scoring, only those deemed highly informative trigger persistence. This approach enables precise control over the persistence path without requiring high-frequency in-memory control planes or cross-node coordination, while preserving unbiasedness of time-aggregated statistics. By integrating approximate statistics from disk-based key-value stores with variance-aware temporal aggregation modeling, the method reduces persistence events by up to 90%, substantially lowering I/O and serialization costs while maintaining or even improving downstream task performance.

Databases continuously evolve through operations such as schema changes, version updates, and data transformations; however, existing approaches typically address these functionalities in isolation, lacking a unified abstraction. This work proposes the first integrated model that unifies continuous schema evolution, version management, and data transformation within a single framework. Built upon general-purpose computational primitives, the model supports operation provenance, conditional update propagation, and change alerts, while employing a declarative mechanism to manage the co-evolution of dependent artifacts—including views and machine learning models. A prototype system implements this framework using an enhanced, parameterized Prolly Tree—a Merkle tree–inspired data structure—to construct a relational-like engine. Experimental evaluation demonstrates that the proposed approach is both feasible and offers tunable performance across diverse evolution scenarios.

This work addresses the absence of a systematic model and unified reference architecture in the current large language model (LLM) inference ecosystem, which hinders accurate prediction of performance, efficiency, and sustainability. To bridge this gap, the paper introduces the first reference architecture for LLM inference ecosystems and presents Kavier, the first cache-aware discrete-event simulation framework capable of efficiently modeling and evaluating system behavior under diverse key-value and prefix caching strategies. Validated against real-world traces, Kavier delivers highly accurate and scalable predictions across dimensions of latency, throughput, and cost, thereby offering critical support for the design, optimization, and operation of LLM inference systems.