Optimizing Time Series Forecasting Architectures: A Hierarchical Neural Architecture Search Approach

2024-06-07

arXiv.org

AI Summary

Manual architecture design and inefficient module composition hinder performance and efficiency in time-series forecasting. Method: This paper proposes Hierarchical Neural Architecture Search (HNAS), a framework that automates model discovery for long-horizon forecasting. Contribution/Results: HNAS introduces (1) a novel hierarchical search space that unifies heterogeneous architectures—including CNNs, RNNs, and Transformers—enabling efficient cross-module composition and task-adaptive search; and (2) a lightweight, differentiable optimization strategy integrated with a modular search space tailored for long-term forecasting. Evaluated on multiple long-horizon benchmarks, models discovered by HNAS reduce parameter count by 30–50% while achieving significantly lower prediction errors than state-of-the-art manually designed baselines. These results demonstrate HNAS’s capability to systematically unlock the potential of deep learning modules through principled, automated architecture discovery.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations

Abstract

The rapid development of time series forecasting research has brought many deep learning-based modules in this field. However, despite the increasing amount of new forecasting architectures, it is still unclear if we have leveraged the full potential of these existing modules within a properly designed architecture. In this work, we propose a novel hierarchical neural architecture search approach for time series forecasting tasks. With the design of a hierarchical search space, we incorporate many architecture types designed for forecasting tasks and allow for the efficient combination of different forecasting architecture modules. Results on long-term-time-series-forecasting tasks show that our approach can search for lightweight high-performing forecasting architectures across different forecasting tasks.

Problem

• Optimizing neural architecture search for time series forecasting
• Combining forecasting modules efficiently via hierarchical design
• Finding lightweight high-performing architectures across forecasting tasks

Innovation

• Hierarchical neural architecture search for time series forecasting
• Efficient combination of different forecasting modules
• Searches lightweight high-performing architectures across tasks