Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence

2024-10-15

arXiv.org

AI Summary

In low-resource settings (only 200 samples), adapting large language models (LLMs) via expert ensembles remains challenging due to reliance on manual hyperparameter tuning and strong prior assumptions. Method: This paper proposes Model Swarms—a parameter-free, assumption-free collaborative search framework grounded in swarm intelligence, operating directly in weight space without fine-tuning. It employs a utility function to guide distributed, diverse expert collaboration, eliminating the need for hyperparameter optimization or predefined ensemble architectures. Contribution/Results: The work introduces the first “utility-driven few-shot swarm collaborative adaptation” paradigm, supporting single-task, multi-task, reward modeling, and personalized adaptation, while enabling spontaneous capability emergence—from weak to strong experts. Experiments demonstrate consistent superiority over 12 baseline ensemble methods across multiple tasks, with up to a 21.0% absolute improvement, empirically validating the emergent nature of new capabilities through collective adaptation.

Technology Category

• Machine Learning: Ensemble Methods
• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

We propose Model Swarms, a collaborative search algorithm to adapt LLMs via swarm intelligence, the collective behavior guiding individual systems. Specifically, Model Swarms starts with a pool of LLM experts and a utility function. Guided by the best-found checkpoints across models, diverse LLM experts collaboratively move in the weight space and optimize a utility function representing model adaptation objectives. Compared to existing model composition approaches, Model Swarms offers tuning-free model adaptation, works in low-data regimes with as few as 200 examples, and does not require assumptions about specific experts in the swarm or how they should be composed. Extensive experiments demonstrate that Model Swarms could flexibly adapt LLM experts to a single task, multi-task domains, reward models, as well as diverse human interests, improving over 12 model composition baselines by up to 21.0% across tasks and contexts. Further analysis reveals that LLM experts discover previously unseen capabilities in initial checkpoints and that Model Swarms enable the weak-to-strong transition of experts through the collaborative search process.

Problem

• Adapting LLM experts via swarm intelligence without tuning
• Optimizing utility functions in low-data regimes effectively
• Enhancing model capabilities through collaborative weight-space search

Innovation

• Collaborative search algorithm using swarm intelligence
• Tuning-free adaptation with diverse LLM experts
• Optimizes utility function in weight space