PACE: Two-Timescale Self-Evolution for Small Language Model Agents

2026-05-21

AI Summary

This work addresses the challenge of enabling efficient self-evolution of frozen small language model (SLM) agents without updating model weights or relying on large language models. The authors propose PACE, a framework employing a dual-timescale evolution mechanism: it first optimizes prompts under a fixed control policy until performance saturates, then introduces high-risk yet empirically validated control policy updates based on a held-out validation set. This approach achieves reliable SLM self-evolution without fine-tuning or external guidance, emphasizing autonomous discovery of task-adapted reasoning strategies. Evaluated across 12 SLM–benchmark combinations, PACE consistently attains state-of-the-art performance, improving base agent efficacy by up to 9.2% and outperforming single-modality evolution baselines by 5.4%, while significantly boosting success rates in multi-turn tool-use tasks.

Abstract

Deploying language-model agents in production often requires substantial compute and human effort to tune prompts, parsers, validators, and other components of the agent pipeline. Self-evolution offers a promising alternative, but most existing frameworks assume access to frontier models that can reliably diagnose failures, propose revisions, and judge their own updates. We study whether frozen small language models (SLMs) can serve as effective self-evolving agents under resource constraints. We propose PACE (Prompt And Control Logic Evolution), a two-timescale framework that coordinates low-risk prompt refinement with higher-risk control-logic updates. PACE evolves prompts under fixed control logic until prompt-level gains saturate, then considers constrained control-logic updates that are accepted through held-out validation. Across three frozen SLM backbones ranging from 4B to 14B parameters and four controlled benchmarks, PACE achieves the best performance on all 12 backbone--benchmark combinations, improving over vanilla SLM agents by up to +9.2% relative improvement and over the stronger single-mode evolution baseline by up to +5.4% relative improvement. A tau-bench case study further shows that PACE improves multi-turn tool-use success over vanilla and prompt-only evolution. These results suggest that reliable SLM agent self-evolution is possible without updating model weights or relying on frontier-model teachers, and that the key benefit is not any single final solver pattern but autonomous, validated discovery of task-appropriate inference strategies.

Problem

• small language models
• self-evolution
• agent optimization
• resource constraints
• frozen models

Innovation

• self-evolution
• small language models
• two-timescale optimization
• prompt refinement
• control logic evolution