Loss Smoothing for Stable Adaptation Under Distribution Shift

2026-07-01

AI Summary

This work addresses the instability caused by abrupt representation shifts when directly optimizing target loss under distribution shift, which often degrades useful features learned on the source task. To mitigate this issue, the authors propose a loss smoothing approach that interpolates between the source and target loss functions during the early phase of adaptation, enabling a gradual transition of the optimization objective. This method introduces, for the first time, a systematic mechanism for smoothing the target objective function. It consistently enhances adaptation stability and performance across diverse scenarios—including supervised distribution shift, vision model transfer, offline-to-online and online reinforcement learning, and language model fine-tuning—while effectively preserving informative source-domain features.

Abstract

In settings such as fine-tuning and reinforcement learning, neural networks are often adapted under distribution shift. Standard adaptation methods typically optimize the target objective directly, inducing an abrupt change from the source training objective. This abrupt transition can distort learned representations, including features that may still be useful for the new task. We investigate whether a more gradual transition can improve adaptation. We propose loss smoothing, a simple approach that interpolates between the source and target training objectives at the start of adaptation. This smooth transition helps to preserve useful features from the source distribution while still enabling the model to specialize to the target distribution. Across controlled supervised shifts, pretrained vision adaptation, offline-to-online and online reinforcement learning, and language model fine-tuning, we find that loss smoothing consistently improves performance, suggesting that smoother objective transitions are a broadly useful tool for model adaptation.

Problem

• distribution shift
• model adaptation
• objective transition
• representation distortion
• neural networks

Innovation

• loss smoothing
• distribution shift
• model adaptation
• objective interpolation
• representation preservation