Deciphering Invariant Feature Decoupling in Source-free Time Series Forecasting with Proxy Denoising

2025-10-07

AI Summary

This work addresses time-series domain adaptation under passive-data scenarios—i.e., adapting a pre-trained model to a sparse target domain without access to source-domain time-series data. To this end, we propose TimePD, the first framework to introduce proxy denoising into this setting, synergistically integrating seasonal-trend decomposition with the strong generalization capability of large language models. TimePD features a dual-branch invariant feature disentanglement architecture and implements bidirectional knowledge distillation at both representation and gradient levels, enabling parameter-free, lightweight domain-invariant modeling. Evaluated on multiple real-world benchmarks, TimePD achieves an average 9.3% improvement over state-of-the-art methods, significantly enhancing forecasting accuracy on sparse target domains. The framework establishes a novel paradigm for compliant, resource-efficient time-series modeling.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

The proliferation of mobile devices generates a massive volume of time series across various domains, where effective time series forecasting enables a variety of real-world applications. This study focuses on a new problem of source-free domain adaptation for time series forecasting. It aims to adapt a pretrained model from sufficient source time series to the sparse target time series domain without access to the source data, embracing data protection regulations. To achieve this, we propose TimePD, the first source-free time series forecasting framework with proxy denoising, where large language models (LLMs) are employed to benefit from their generalization capabilities. Specifically, TimePD consists of three key components: (1) dual-branch invariant disentangled feature learning that enforces representation- and gradient-wise invariance by means of season-trend decomposition; (2) lightweight, parameter-free proxy denoising that dynamically calibrates systematic biases of LLMs; and (3) knowledge distillation that bidirectionally aligns the denoised prediction and the original target prediction. Extensive experiments on real-world datasets offer insight into the effectiveness of the proposed TimePD, outperforming SOTA baselines by 9.3% on average.

Problem

• Adapting pretrained forecasting models without source data access
• Addressing sparse target domain time series forecasting challenges
• Leveraging LLMs while calibrating systematic biases via proxy denoising

Innovation

• Proxy denoising calibrates LLM biases dynamically
• Dual-branch learning enforces invariant feature disentanglement
• Bidirectional knowledge distillation aligns denoised predictions