AgentR A Stateful and Recovery-Aware Software Architecture for LLM-based Auditable Workflows

2026-08-15

AI Summary

This study addresses the critical challenges of state persistence and auditability in Large Language Model (LLM) applications by proposing AgentR, a stateful architecture. AgentR introduces a novel persistent state machine tailored for LLM workflows, integrating ACID-compliant cost logging and orphan task detection mechanisms. Through asynchronous orchestration, the system enables robust fault recovery and comprehensive end-to-end auditing. Experimental evaluations demonstrate that AgentR achieves a 99.2% task completion rate and a 4.3× parallel speedup. These results effectively validate the significant advantages of stateful design in enhancing the reliability, observability, and accountability of LLM-based systems.

Abstract

Modern LLM-based applications increasingly require multi- stage execution, persistent intermediate state, retry seman- tics, and auditable usage accounting. However, many LLM applications are still implemented as stateless prompt- response wrappers or session-bounded conversational sys- tems, which makes them difficult to recover, audit, and re- produce after interruption or failure. We propose AgentR, a stateful architecture for LLM workflow systems that en- ables persistence and recovery, instantiated through scien- tific literature review as a representative use case. AgentR represents research intent, generated queries, candidate- paper assessments and gap analyses as durable workflow artifacts, and executes the pipeline through asynchronous BullMQ workers backed by Redis, with PostgreSQL as the persistence store. The design includes explicit processing state transitions, retries with exponential backoff, orphan job detection, credit-aware pre-checks, ACID token-cost logging, and Type-2 slowly changing pricing records. We evaluate AgentR on telemetry collected from a prototype deployment. At the LLM stage, the system achieves 99.2% job completion, and mean latencies of 9.0 s, 18.9 s, and 25.4 s for intent decomposition, query generation, and paper scoring, respectively. Parallel scoring allows for analytical latency modeling from observed calls, leading to as much as 4.3 wall-clock speedup over sequential execution. The results provide preliminary proof-of-concept that persistent state machine design, asynchronous orchestration, and cost-aware usage logging can enable improved observability, recoverability, and operational accountability in LLM workflow systems. The prototype implementation of AgentR is publicly available at: https://github.com/ RiyaSamanta/AgentR-public.

Problem

• LLM-based applications
• stateless architecture
• recoverability
• auditability
• workflow persistence

Innovation

• Stateful Architecture
• Recovery-Aware Workflow
• Durable Artifacts
• Asynchronous Orchestration
• Auditable Accounting