Problem
Research questions and friction points this paper is trying to address.
runtime harnesses
iterative search
overfitting
failure diagnosis
Ecdysis: Efficient and Effective Training of Runtime Harnesses for LLM Agents
🤖 AI Summary
为解决自演化运行时框架在优化大语言模型代理时的效率低下和过拟合问题，Ecdysis通过跨任务失败模式分析和故障驱动协作改进方法有效区分模型特性和系统缺陷，从而提高训练效率和推理准确性。
📝 Abstract
Self-evolving runtime harnesses can substantially improve the capabilities of large language model (LLM) agents and provide a promising paradigm for optimizing agent execution. Existing harness evolution methods typically rely on iterative search, repeatedly evaluating and revising candidate harnesses based on execution feedback from task instances. While this paradigm enables continuous harness optimization, it incurs substantial time overhead due to repeated agent executions and code modifications, and may overfit to observed tasks and specific failure patterns, resulting in degraded generalization to unseen tasks. We identify the lack of principled failure diagnosis as a key bottleneck in harness evolution: an observed failure can reflect either model-specific deficiencies or systematic harness deficiencies, and directly optimizing against individual failures can lead to unnecessary model-specific accommodation. We therefore propose Ecdysis, an efficient and effective framework that distinguishes model-specific accommodation from harness-level repair and biases adaptation toward systematic harness deficiencies by identifying recurring cross-task failure patterns. Ecdysis adopts a batch-level cross-instance failure aggregation paradigm to jointly analyze failure evidence from multiple task instances and further introduces Failure-Driven Collaborative Refinement to diagnose failure causes and iteratively refine harness modification specifications. By combining cross-instance failure analysis with multi-role diagnosis, Ecdysis enables more effective harness evolution with lower training time. Experiments show that Ecdysis achieves up to a 1.84x speedup in harness training compared with existing harness evolution methods, while improving the reasoning accuracy of the resulting harnesses by 18.56%.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Failure-Driven Collaborative Refinement
Cross-Instance Failure Aggregation
Harness Evolution
🔎 Similar Papers
No similar papers found.
