AutoTrainess: Teaching Language Models to Improve Language Models Autonomously

2026-06-30

AI Summary

Current post-training of language models remains heavily reliant on manual intervention and lacks automation and continuous optimization capabilities. This work proposes an autonomous post-training framework grounded in an agent-computer interface, which externalizes human training expertise into structured workflows, rules, and constraints. The framework integrates modules for planning, data construction, stable training scheduling, evaluation, and state persistence to enable end-to-end autonomous iterative optimization. Evaluated on PostTrainBench, the method achieves an average score of 26.94 (GPT-5.4), substantially outperforming the CLI baseline (23.21). Furthermore, it successfully enhances DeepSeek-V4-Flash’s performance from 12.13 to 19.58, demonstrating its reliability and generalization capability in complex training scenarios.

Abstract

Training language models (LMs) remains a highly human-intensive process, even as frontier language model agents become increasingly capable at software engineering and other long-horizon tasks. A central challenge is that autonomous post-training is not just a coding problem: it requires the agent to repeatedly plan iterations, construct benchmark-aligned data, run stable training jobs, evaluate checkpoints, and preserve experiment state across many hours of interaction. We present AutoTrainess, a LM agent that exposes these operations as a repository of agent-computer interfaces for planning, data preparation, training, evaluation, and logging. Rather than leaving the agent to operate in a raw CLI environment with an underspecified action space, AutoTrainess externalizes prior human experience as explicit workflows, rules, and execution constraints that guide the agent toward effective and reliable training behavior. On PostTrainBench, AutoTrainess consistently outperforms CLI-only baselines, achieving 26.94 average score with GPT-5.4 (Codex) versus 23.21 for CLI-only. It also generalizes across models and harnesses, improving DeepSeek-V4-Flash (OpenCode) from 12.13 to 19.58.

Problem

• autonomous post-training
• language models
• human-intensive process
• training automation
• benchmark-aligned data

Innovation

• autonomous post-training
• agent-computer interface
• language model training automation
• workflow-guided agent
• PostTrainBench