Ground-Truth Neighborhood Regularization for Reinforcement Learning Post-Training of Time Series Foundation Models

2026-08-08

AI Summary

This work identifies and formally characterizes a previously overlooked issue in time series foundation models: when fine-tuned with reinforcement learning (RL), these models often suffer from “suboptimal collapse,” wherein the output distribution deviates significantly from the ground truth in certain prediction regions. To address this, the authors propose a novel regularization method that encourages the model to concentrate its probability mass within a neighborhood of the true values during RL-based fine-tuning. This mechanism is designed to be algorithm-agnostic and can be seamlessly integrated into various RL frameworks to effectively constrain the optimization trajectory. Experimental results demonstrate that the proposed approach substantially mitigates suboptimal collapse, leading to consistent improvements in both predictive accuracy on downstream tasks and the generation quality of high-fidelity samples.

Abstract

Time series forecasting (TSF) plays an important role in a wide range of real-world applications. Recently, time series foundation models (TSFMs), pretrained on large-scale datasets, have demonstrated strong generalization capabilities and emerged as an important paradigm for TSF. Reinforcement learning (RL) post-training has consequently attracted growing attention as a means of further improving their performance on downstream tasks. However, we find that, in certain forecast regions, RL post-training may gradually shift the output distributions of TSFMs away from the ground truth, thereby limiting their performance. We refer to this phenomenon as \textbf{suboptimal collapse}. Our analysis suggests that difficulty in initially sampling high-quality trajectories near the ground truth is an important contributing factor to suboptimal collapse. To address this issue, we propose Ground-Truth Neighborhood Regularization (GTN-R) for RL post-training of TSFMs. GTN-R uses the ground truth as a reference for locating high-quality regions and guides the model's probability mass toward the ground-truth neighborhood. This increases the probability of sampling high-quality trajectories, mitigates suboptimal collapse, and improves performance. Moreover, GTN-R can be flexibly integrated into various RL methods for TSFMs. Extensive experiments show its effectiveness.

Problem

• suboptimal collapse
• rein日晚间 learning post-training
• time series foundation models
• output distribution shift
• ground truth deviation

Innovation

• Ground-Truth Neighborhood Regularization
• Reinforcement Learning Post-Training
• Time Series Foundation Models
• Suboptimal Collapse
• Trajectory Sampling