MERA: Model Evolution and Routing with Skill Adaptation for Agentic Systems at Scale

2026-08-10

AI Summary

This work addresses the limited cost-effectiveness of existing routing methods that merely assign simple tasks to small models without enhancing their capabilities. To overcome this, the authors propose a multi-cycle adaptation mechanism operating at the granularity of single inference calls. The approach leverages a teacher model to generate verification demonstrations from the small model’s failures, integrating skill distillation and LoRA fine-tuning to continuously improve its competence. Joint optimization is performed over a dynamic skill library, task-specific adapters, and a cost-calibrated routing policy, complemented by a verifier-supported fallback mechanism. Experiments show that Qwen2.5-Coder-1.5B achieves a pass rate increase from 28.7% to 49.7% on HumanEval+MBPP; the deployment strategy attains 88.3% of peak performance at only 60.8% of the cost; and Qwen3.5-2B matches the performance of an unadapted 4B model on TAU-2.

Abstract

LLM agents execute heterogeneous sequences of model calls within a single task: some invocations require careful reasoning, while others are structured steps such as formatting or tool-argument construction. Prior routing methods exploit this asymmetry by assigning easy invocations to a cheaper small model and difficult ones to a large model. Such policies reduce inference cost, but they leave the small model's capability unchanged, so attainable savings remain bounded by the work the student can already solve. MERA instead improves the small model itself, using a single model invocation as the unit of adaptation. In each cycle, MERA replays failed student invocations to obtain execution-verified teacher demonstrations, distills recurring procedures into an iteratively updated SkillBook, and fine-tunes a student LoRA adapter via supervised learning and optional GRPO. Routing serves as supporting machinery for deployment: the improved student is served behind a cost-calibrated router with verifier-backed fallback, and a candidate SkillBook, adapter, or router is admitted only when joint replay preserves task quality. Empirically, four-cycle adaptation raises Qwen2.5-Coder-1.5B from 28.7% to 49.7% pass on held-out HumanEval+MBPP. Under verifier-backed fallback, the deployed policy retains 88.3% pass at 60.8% of always-Luna cost. On TAU-2, a fine-tuned Qwen3.5-2B improves from 14/35 to 18/35 and matches an unadapted 4B model. These results indicate that verifier-backed multi-cycle adaptation can increase small-model capability, rather than only routing around a fixed student.

Problem

• model routing
• small model capability
• cost-efficient inference
• agentic systems
• capability adaptation

Innovation

• model evolution
• skill adaptation
• LoRA fine-tuning
• verifier-backed fallback
• multi-cycle distillation