Train Separately, Merge Together: Modular Post-Training with Mixture-of-Experts

2026-04-20

AI Summary

This work addresses the high cost, poor scalability, and catastrophic forgetting associated with conventional monolithic post-training when extending large language models to new domains. The authors propose BAR, a modular post-training framework in which domain-specific experts—trained independently through intermediate training, supervised fine-tuning, and reinforcement learning—are integrated into a Mixture-of-Experts architecture via a lightweight routing mechanism. BAR enables independent expert updates, linearly scalable training costs, and mitigation of performance degradation across tasks. Evaluated on a 7B-parameter model spanning mathematics, code generation, tool usage, and safety, the approach achieves a composite score of 49.1, matching or surpassing traditional full retraining baselines while substantially reducing update complexity and computational overhead.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Extending a fully post-trained language model with new domain capabilities is fundamentally limited by monolithic training paradigms: retraining from scratch is expensive and scales poorly, while continued training often degrades existing capabilities. We present BAR (Branch-Adapt-Route), which trains independent domain experts, each through its own mid-training, supervised finetuning, and reinforcement learning pipeline, and composes them via a Mixture-of-Experts architecture with lightweight router training. Unlike retraining approaches that mix all domains and require full reprocessing for any update (with cost scaling quadratically), BAR enables updating individual experts independently with linear cost scaling and no degradation to existing domains. At the 7B scale, with experts for math, code, tool use, and safety, BAR achieves an overall score of 49.1 (averaged across 7 evaluation categories), matching or exceeding re-training baselines (47.8 without mid-training, 50.5 with). We further show that modular training provides a structural advantage: by isolating each domain, it avoids the catastrophic forgetting that occurs when late-stage RL degrades capabilities from earlier training stages, while significantly reducing the cost and complexity of updating or adding a domain. Together, these results suggest that decoupled, expert-based training is a scalable alternative to monolithic retraining for extending language models.

Problem

• modular post-training
• Mixture-of-Experts
• catastrophic forgetting
• domain extension
• language model scaling

Innovation

• Modular Post-Training
• Mixture-of-Experts
• Catastrophic Forgetting
• Domain Experts
• Linear Cost Scaling