Data Augmentation Policy Search for Long-Term Forecasting

2024-05-01

arXiv.org

AI Summary

This paper addresses the inefficiency and poor generalization of manually designed data augmentation strategies in long-horizon time series forecasting. To this end, we propose TSAA—the first automated augmentation method tailored for long-term time series prediction. Methodologically, TSAA introduces automatic augmentation to this task via a lightweight two-stage optimization framework: (i) a limited pre-training phase, followed by (ii) Bayesian optimization-driven policy search—bypassing costly end-to-end bilevel optimization. Crucially, TSAA explicitly models a time-series-specific transformation space, incorporating operations such as temporal shifting, scaling, and masking. Extensive experiments on multiple univariate and multivariate long-horizon forecasting benchmarks demonstrate that TSAA consistently outperforms strong baselines—including Autoformer and Informer—with average MAE reductions of 3.2%–7.8%. The implementation is publicly available.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Studies of user behavior, including longitudinal effects of personalized systems

Abstract

Data augmentation serves as a popular regularization technique to combat overfitting challenges in neural networks. While automatic augmentation has demonstrated success in image classification tasks, its application to time-series problems, particularly in long-term forecasting, has received comparatively less attention. To address this gap, we introduce a time-series automatic augmentation approach named TSAA, which is both efficient and easy to implement. The solution involves tackling the associated bilevel optimization problem through a two-step process: initially training a non-augmented model for a limited number of epochs, followed by an iterative split procedure. During this iterative process, we alternate between identifying a robust augmentation policy through Bayesian optimization and refining the model while discarding suboptimal runs. Extensive evaluations on challenging univariate and multivariate forecasting benchmark problems demonstrate that TSAA consistently outperforms several robust baselines, suggesting its potential integration into prediction pipelines. Code is available at this repository: https://github.com/azencot-group/TSAA.

Problem

• Develops TSAA for time-series data augmentation
• Addresses long-term forecasting overfitting in neural networks
• Utilizes Bayesian optimization for augmentation policy search

Innovation

• Time-series automatic augmentation
• Bilevel optimization process
• Bayesian optimization policy search