Agent JIT Compilation for Latency-Optimizing Web Agent Planning and Scheduling

2026-05-20

AI Summary

This work addresses the high latency and tool misuse inherent in current agents that rely on repeated large language model (LLM) invocations for natural language tasks. It introduces, for the first time, just-in-time (JIT) compilation principles into web agent planning, proposing JIT-Planner and JIT-Scheduler. JIT-Planner directly compiles task descriptions into verified, low-cost executable code plans, while JIT-Scheduler optimizes parallel execution through Monte Carlo-based scheduling. To enhance correctness, the approach incorporates a formal tool protocol constrained by state invariants. Experimental results across five web applications demonstrate that JIT-Planner achieves a 10.4× speedup over Browser-Use with a 28% improvement in accuracy, and JIT-Scheduler yields a 2.4× speedup over OpenAI CUA with a 9% gain in accuracy.

Abstract

Computer-use agents (CUA) automate tasks specified with natural language such as "order the cheapest item from Taco Bell" by generating sequences of calls to tools such as click, type, and scroll on a browser. Current implementations follow a sequential fetch-screenshot-execute loop where each iteration requires an LLM call, resulting in high latency and frequent errors from incorrect tool use. We present agent just-in-time (JIT) compilation, an alternative that compiles task descriptions directly into executable code that is free to include LLM calls, tool calls, and parallelization. Our approach comprises three components: (1) JIT-Planner, which generates multiple code plans, validates each against tool specifications, and selects the minimum-cost candidate; (2) JIT-Scheduler, which explores parallelization strategies via Monte Carlo cost estimation from learned latency distributions; and (3) an invariant-enforcing tool protocol specifying precondition and postcondition state requirements that reduce the rate of generating plans with incorrect tool use. Across 5 web applications, JIT-Planner achieves $10.4\times$ speedup and $+28\%$ accuracy over Browser-Use, while JIT-Scheduler achieves $2.4\times$ speedup and $+9\%$ accuracy over OpenAI CUA.

Problem

• computer-use agents
• latency
• tool use errors
• web automation
• sequential execution

Innovation

• just-in-time compilation
• agent planning
• parallelization
• tool protocol
• latency optimization