Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift

2024-10-13

arXiv.org

AI Summary

Real-world time series exhibit significant segment-level distribution shifts—e.g., due to seasonality, operational conditions, or semantic changes—rendering single-model approaches insufficient for generalization. To address this, we propose TFPS, the first framework featuring a pattern-aware dynamic expert routing mechanism. TFPS jointly encodes local segments in both time and frequency domains, then applies subspace clustering to automatically identify heterogeneous temporal patterns; each identified pattern activates a dedicated expert network for prediction. This design explicitly models and adaptively evolves segment-level temporal dynamics, overcoming the generalization bottleneck inherent in conventional single-model paradigms. Extensive experiments on multiple real-world benchmarks demonstrate that TFPS consistently outperforms state-of-the-art methods, with particularly notable gains in long-horizon forecasting. The code and datasets are publicly available.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Planning, Routing, and Scheduling: Temporal Planning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Time series forecasting, which aims to predict future values based on historical data, has garnered significant attention due to its broad range of applications. However, real-world time series often exhibit complex non-uniform distribution with varying patterns across segments, such as season, operating condition, or semantic meaning, making accurate forecasting challenging. Existing approaches, which typically train a single model to capture all these diverse patterns, often struggle with the pattern drifts between patches and may lead to poor generalization. To address these challenges, we propose extbf{TFPS}, a novel architecture that leverages pattern-specific experts for more accurate and adaptable time series forecasting. TFPS employs a dual-domain encoder to capture both time-domain and frequency-domain features, enabling a more comprehensive understanding of temporal dynamics. It then uses subspace clustering to dynamically identify distinct patterns across data patches. Finally, pattern-specific experts model these unique patterns, delivering tailored predictions for each patch. By explicitly learning and adapting to evolving patterns, TFPS achieves significantly improved forecasting accuracy. Extensive experiments on real-world datasets demonstrate that TFPS outperforms state-of-the-art methods, particularly in long-term forecasting, through its dynamic and pattern-aware learning approach. The data and codes are available: url{https://github.com/syrGitHub/TFPS}.

Problem

• Addressing patch-level distribution shifts in time series forecasting
• Handling non-uniform patterns across time series segments
• Improving generalization against pattern drifts in temporal data

Innovation

• Pattern-specific experts for time series forecasting
• Dual-domain encoder captures time and frequency features
• Subspace clustering dynamically identifies distinct patterns