Compression, structure, and executor capability: a controlled real-cost decomposition of language-model agent skill optimisation

2026-07-03

AI Summary

This study addresses the lack of systematic empirical evidence on whether existing skill optimization strategies for language model agents—such as compression and structuring—can simultaneously reduce costs and maintain task success rates. Within a unified experimental framework, the authors conduct 1,200 controlled trials across 40 software engineering tasks under 10 distinct skill representation conditions. For the first time, they disentangle the effects of skill compression, structuring, scope-aware loading, and executor-level factors, evaluating performance through both real API costs and task pass rates. The results demonstrate that enhancing only the executor capability yields a substantial 27-percentage-point improvement in task success (at approximately five times the cost), whereas none of the other optimization strategies achieve an effective trade-off between quality and cost across any executor configuration.

Abstract

Agent skills, reusable instruction artefacts supplied to a tool-using language model, are increasingly optimised by shortening, structural rewriting, stronger-model compilation, and scoped loading, on the assumption that a smaller or better-organised skill lowers cost while preserving success. That assumption is rarely tested with quality and real monetary cost measured on the same runs and the contributing factors separated. This study reports a controlled decomposition over ten skill-delivery conditions, 40 software-engineering tasks, and three repetitions per cell (1,200 rollouts), separating no-skill execution, raw skills, deterministic shortening, linear and structured rendering from a shared semantic ledger, scoped loading, and the compiler and executor model tiers. Quality is the verifier pass rate at task level; cost is solve-stage token cost at standard provider prices, with a token-volume-normalised view for robustness and compilation cost amortised separately. The task is the unit of inference, intervals are task-clustered, and the contrast family is multiplicity-controlled. Deterministic shortening is close to the raw baseline but does not establish non-inferiority within the preset margin. Structured rendering and scoped loading lower pass rate on the compact executor without lowering cost, and structured rendering is indistinguishable from linear text at matched content. The only contrast surviving correction is executor capability, which raises pass rate by 27 percentage points at roughly five times the real cost, with compiler tier showing no robust effect. Under real prices no optimised representation reaches a practical break-even. The evidence indicates that executor capability is the dominant lever and that no representation strategy improves over the raw skill on either executor tier.

Problem

• skill optimisation
• language-model agent
• cost-quality trade-off
• structured representation
• executor capability

Innovation

• skill optimization
• controlled decomposition
• executor capability
• real-cost evaluation
• language-model agents