Discovering Novel LLM Experts via Task-Capability Coevolution

2026-04-16

AI Summary

This work addresses the limitations of conventional large language model (LLM) training, which relies on static datasets or fixed reward signals and struggles to continuously expand diverse capabilities. The authors propose the AC/DC framework, which introduces an open-ended co-evolutionary mechanism into LLM development for the first time. By dynamically co-evolving tasks and model competencies within a single training run, AC/DC automatically discovers expert models endowed with novel skills. The approach integrates model merging with natural language–based task generation to synthesize data, thereby sustaining evolving populations of both LLMs and tasks without explicit optimization on downstream benchmarks. The resulting expert models exhibit broader capability coverage than substantially larger models, achieve significant reductions in GPU memory consumption, and demonstrate continual performance gains in multi-agent selection settings.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Frontier model developers aim to train models continually to possess emergent, diverse capabilities. To extend capabilities, the current pre-training and post-training paradigm requires manually starting training runs with static datasets or reward functions every time. Addressing this limitation, our work pursues the insight that open-endedness (via the coevolution of models and tasks) can discover models with increasingly novel skills in a single run. We introduce a new model development framework that extends coevolution to large language model (LLM) discovery, open-ended \textit{Assessment Coevolving with Diverse Capabilities} (AC/DC). AC/DC evolves both LLMs via model merging and natural language tasks via synthetic data generation. AC/DC discovers growing archives of LLMs that surpass the capabilities of larger LLMs while taking up less GPU memory. In particular, our LLM populations achieve a broader Coverage of expertise than other curated models or baselines on downstream benchmarks, without \textit{any} explicit benchmark optimization. Furthermore, AC/DC improves Coverage over time, continually innovates on tasks and models, and improves performance in multi-agent best-of-N selection. Our findings highlight the potential of coevolution as a means of discovering broader sets of capabilities from base LLMs. Overall, AC/DC brings us one step closer to a profoundly new paradigm of LLM development, where continual improvements to the diversity of model capabilities can be accelerated by leveraging existing models as stepping stones to increasingly powerful models.

Problem

• large language models
• open-endedness
• coevolution
• task-capability
• model discovery

Innovation

• coevolution
• model merging
• synthetic data generation
• open-endedness
• capability coverage