🤖 AI Summary
This study addresses the limitations of standard Transformers in financial time series forecasting, where performance is hindered by noise interference, short-term memory dynamics, and distribution shifts. To overcome these challenges, the authors propose an enhanced Transformer architecture that integrates cosine annealing with warm restarts for learning rate scheduling and introduces a novel Shifted Data Augmentation (SDA) technique. SDA substantially reduces prediction errors and training instability, demonstrating superior robustness compared to merely increasing model complexity. Empirical evaluations on the VN30 and S&P 500 datasets show that the proposed method achieves state-of-the-art performance in terms of prediction accuracy, training stability, and robustness to hyperparameter variations.
📝 Abstract
Transformers have shown remarkable success in sequence modeling, yet their direct application to financial time series remains challenging due to noisy signals, short-memory dynamics, and distributional shifts. This paper proposes a modified Transformer architecture for one-step stock index forecasting, combined with advanced learning-rate scheduling and a novel Shifted Data Augmentation (SDA) technique. We evaluate the proposed framework on two benchmark stock index datasets, VN30 and S&P 500. Experimental results demonstrate that cosine annealing with warmup consistently improves forecasting accuracy over the generalized inverse-power scheduler. Furthermore, SDA substantially reduces forecasting errors and run-to-run variability while improving robustness to hyperparameter selection. The combination of cosine annealing scheduling and SDA achieved the best performance on both datasets, indicating that data augmentation can play a more important role than increasing model complexity in Transformer-based financial forecasting. These findings provide a practical and computationally efficient approach for robust stock index forecasting in noisy financial environments.