xPatch: Dual-Stream Time Series Forecasting with Exponential Seasonal-Trend Decomposition

2024-12-23

AI Summary

To address the challenges of modeling long-range dependencies and seasonality-trend coupling in long-term time series forecasting with Transformers, this paper proposes xPatch, a dual-stream architecture. Methodologically: (1) it introduces a novel exponential seasonality-trend decomposition module to explicitly disentangle multi-scale temporal dynamics; (2) it designs a synergistic modeling mechanism comprising a linear MLP stream and a nonlinear CNN stream, operating on channel-wise independent patches; (3) it adopts an arctangent loss function to mitigate gradient imbalance and integrates a sigmoid learning rate scheduler to enhance training stability. Extensive experiments on multiple benchmark datasets demonstrate that xPatch consistently outperforms state-of-the-art Transformer-based and linear forecasting models, achieving an average 12.7% improvement in long-horizon prediction accuracy while effectively suppressing overfitting—validating its superior capacity to capture complex temporal patterns.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

In recent years, the application of transformer-based models in time-series forecasting has received significant attention. While often demonstrating promising results, the transformer architecture encounters challenges in fully exploiting the temporal relations within time series data due to its attention mechanism. In this work, we design eXponential Patch (xPatch for short), a novel dual-stream architecture that utilizes exponential decomposition. Inspired by the classical exponential smoothing approaches, xPatch introduces the innovative seasonal-trend exponential decomposition module. Additionally, we propose a dual-flow architecture that consists of an MLP-based linear stream and a CNN-based non-linear stream. This model investigates the benefits of employing patching and channel-independence techniques within a non-transformer model. Finally, we develop a robust arctangent loss function and a sigmoid learning rate adjustment scheme, which prevent overfitting and boost forecasting performance. The code is available at the following repository: https://github.com/stitsyuk/xPatch.

Problem

• Time Series Prediction
• Transformer Models
• Seasonality and Trend Analysis

Innovation

• xPatch model
• Dual-channel approach
• Exponential decomposition method