Learning to Select, Not Relearn: Hard-Routed Mixtures of Reasoning LoRAs

2026-06-30

AI Summary

This work addresses the challenge of effectively combining multiple frozen LoRA inference experts for multi-domain adaptation without sharing raw data, while avoiding the violation of LoRA’s unit-scale update assumption caused by soft routing. The authors propose Hard-Routed MoR-LoRA, a two-stage framework: first, domain-specific LoRA experts are independently trained via reinforcement learning and then frozen; second, only a lightweight shared router and small attention LoRAs are trained, employing hard top-1 routing to assign each token to a single expert. This approach introduces hard routing into frozen LoRA composition for the first time, preserving the original expert behaviors and revealing that soft routing often degenerates to selecting a single expert in practice. Experiments across five benchmarks, diverse model scales, and architectures demonstrate that the method significantly reduces trainable parameters while matching or surpassing soft-routing baselines in inference performance.

Abstract

Composing independently trained LoRA adapters into a single large language model is useful for multi-domain adaptation, especially when the original training data cannot be shared. A common approach is to use MoE-style routing over LoRA experts, but for frozen pretrained adapters, soft weighted combinations can change the unit-scale additive update under which each LoRA module was originally trained. We propose \textbf{Hard-Routed MoR-LoRA}, a two-stage framework for composing frozen reasoning LoRA experts through unit-scale hard selection. First, domain-specific LoRA adapters are trained independently using reinforcement learning from verifiable feedback to obtain reasoning experts. Then, all experts are frozen, reasoning traces are distilled from them, and only a lightweight shared router together with a small attention LoRA is trained for integration. The router selects exactly one expert per token using hard top-1 routing, while a straight-through estimator enables gradient-based training. Experiments across five benchmarks, multiple model scales, and additional model families show that Hard-Routed MoR-LoRA preserves expert behavior while requiring substantially fewer trainable parameters than soft-routing mixture baselines. Our analysis further shows that normalized soft mixtures often concentrate most routing mass on a single expert, suggesting that hard unit-scale routing provides a simple and efficient abstraction for frozen LoRA expert composition.

Problem

• LoRA
• Mixture of Experts
• Hard Routing
• Frozen Adapters
• Multi-domain Adaptation

Innovation

• Hard-Routed MoR-LoRA
• LoRA composition
• hard routing
• frozen adapters
• reasoning experts