DynaMiCS: Fine-tuning LLMs with Performance Constraints using Dynamic Mixtures

2026-05-11

AI Summary

This work addresses the challenge of multi-domain fine-tuning of large language models, which requires enhancing performance on target domains while strictly preserving capabilities in constrained domains—such as general knowledge retention, instruction following, or safety. The authors formulate this as a constrained optimization problem and propose a dynamic data mixing strategy that operates without a reference model, per-sample scoring, or manual hyperparameter tuning. Their method constructs a cross-domain influence slope matrix using short-horizon probing signals and computes optimal mixing weights by solving a constrained optimization over the probability simplex to satisfy explicit performance constraints. Experiments across diverse multi-domain fine-tuning scenarios demonstrate that this approach achieves significantly better target-domain gains and higher constraint satisfaction rates than fixed-mix baselines, all at reduced computational cost.

Abstract

Multi-domain fine-tuning of large language models requires improving performance on target domains while preserving performance on constrained domains, such as general knowledge, instruction following, or safety evaluations. Existing data mixing strategies rely on fixed heuristics or adaptive rules that cannot explicitly enforce preservation of such capabilities. We propose DynaMiCS, a dynamic mixture optimizer that casts multi-domain fine-tuning as a constrained optimization problem. At each update, DynaMiCS performs short domain-specific probing runs to estimate a slope matrix of local cross-domain effects, capturing how training on each fine-tuning dataset affects each evaluation domain. These estimates are then used to compute mixture weights through optimization over the probability simplex, with the objective of improving target-domain performance while keeping constrained-domain losses below reference levels. Across multi-domain fine-tuning scenarios with varying numbers of target and constrained domains, DynaMiCS achieves stronger target-domain improvements and higher constraint satisfaction than fixed-mixture baselines, at lower computational cost and without reference models, per-example scoring, or manually tuned mixture weights.

Problem

• multi-domain fine-tuning
• performance constraints
• large language models
• constrained optimization
• capability preservation

Innovation

• constrained optimization
• dynamic mixture
• multi-domain fine-tuning
• cross-domain effects
• LLM alignment