(How) Learning Rates Regulate Catastrophic Overtraining

2026-04-15

AI Summary

This study addresses the issue of catastrophic forgetting in supervised fine-tuning (SFT) of large language models, which often undermines their foundational capabilities. Through theoretical analysis and empirical validation, the work investigates how learning rate acts as an implicit regularizer, shaping optimization trajectories and model sharpness to induce forgetting and overfitting. The authors demonstrate for the first time that although fine-tuning with large or small learning rates can yield comparable loss values, they converge to fundamentally different solutions. Notably, learning rate decay significantly increases the sharpness of pretrained models, thereby exacerbating catastrophic forgetting. By establishing a deep connection between optimization dynamics in pretraining and fine-tuning, this work reveals a novel mechanism for mitigating overfitting during adaptation.

Technology Category

• Natural Language Processing: Learning & Optimization for NLP
• Machine Learning: Learning Theory
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Supervised fine-tuning (SFT) is a common first stage of LLM post-training, teaching the model to follow instructions and shaping its behavior as a helpful assistant. At the same time, SFT may harm the fundamental capabilities of an LLM, particularly after long pretraining: a phenomenon known as catastrophic overtraining (Springer et al., 2025). To understand overtraining, we first investigate catastrophic forgetting in finetuning through the lens of implicit regularization of the learning rate. For models trained to the same SFT loss, we identify how the learning rate mediates optimization: finetuning with large and small steps converges to qualitatively different models. Next, we link forgetting to overtraining: learning rate decay increases the sharpness of the pretrained model, which in turn exacerbates catastrophic forgetting during SFT, leading to overtraining. Our findings paint a picture of the overtraining mechanism in LLMs and broadly contribute to the understanding of the interplay between optimization dynamics during pretraining and finetuning.

Problem

• catastrophic overtraining
• supervised fine-tuning
• learning rate
• catastrophic forgetting
• LLMs

Innovation

• learning rate
• catastrophic overtraining
• implicit regularization
• loss sharpness
• supervised fine-tuning