TraceLab: Characterizing Coding Agent Workloads for LLM Serving

2026-06-29

AI Summary

This work addresses the lack of realistic coding agent workload data in current large language model (LLM) serving systems, which hinders efficient optimization. We present the first large-scale trace dataset from real-world coding agents, comprising approximately 4,300 sessions, 350,000 LLM inference steps, and 430,000 tool invocations. The dataset reveals key runtime characteristics in multi-agent, multi-model scenarios, including long autonomous loops, short outputs paired with long contexts, a heavy-tailed tool invocation distribution, and high—but imperfect—prefix cache hit rates. Leveraging these insights, we propose targeted optimizations: low-overhead tool invocation, context-aware prefilling, semantic tool latency prediction, and improved KV cache management. Our findings provide empirical grounding and new directions for designing efficient LLM serving systems.

Abstract

Coding agents are rapidly becoming a major application of agentic LLMs, but serving them efficiently remains challenging. Progress on this challenge requires understanding real workload patterns, yet the data needed for such analysis is largely absent. Existing public traces and benchmarks do not capture real, day-to-day coding-agent usage across multiple agents and model families for serving-system analysis. To help fill this gap, we collect and release a trace of roughly 4,300 coding-agent sessions, containing about 350,000 LLM steps and 430,000 tool calls from our own day-to-day use of Claude Code and Codex. Our analysis shows that coding-agent workloads feature long autonomous loops, long contexts with short outputs, diverse and heavily-tailed tool calls, and high but imperfect prefix cache hit rates. These findings point to concrete opportunities for optimizing serving, including lower-overhead tool calling, append-length-aware prefill, semantic-aware tool-latency prediction, and improved KV-cache management around human-paced gaps. We release the dataset, trace collection pipeline, and analysis code at https://github.com/uw-syfi/TraceLab.git; the project website is https://tracelab.cs.washington.edu.

Problem

• coding agents
• LLM serving
• workload characterization
• trace data
• serving optimization

Innovation

• coding agents
• LLM serving
• workload characterization
• tool calling
• KV-cache optimization