EVOTS: Evolutionary Transformer Search for Time Series Forecasting

2026-06-30

AI Summary

This work addresses the lack of task adaptability in Transformer architectures for multivariate time series forecasting by proposing a neural architecture search framework based on evolutionary algorithms. The approach employs a modular gene encoding scheme to flexibly compose attention, feed-forward, and projection components, and incorporates a structure repair mechanism to guarantee the validity of generated models. It is the first to apply evolutionary search systematically to the design of time series Transformers, enabling automatic discovery of task-specific efficient architectures without handcrafted rules. Experiments on the ETT benchmark datasets demonstrate that the evolved models achieve competitive or superior performance against strong baselines across multiple metrics in multivariate-to-multivariate forecasting settings, while maintaining reasonable computational overhead during the search process.

Abstract

Evolutionary neural architecture design for multivariate time-series forecasting remains underexplored, with most approaches relying on fixed Transformer architectures despite substantial variation across tasks and forecasting settings. This paper introduces an evolutionary neural architecture search framework for discovering task-adaptive Transformer-like models for time-series forecasting (EVOTS). Architectures are encoded using a modular genome representation that enables flexible composition of attention, feed-forward, and projection components, while a repair mechanism enforces structural validity throughout the evolutionary process. This formulation allows effective exploration of a diverse architecture space without relying on hand-crafted design rules. The proposed approach is evaluated on four benchmark datasets from the ETT family (ETTh1, ETTh2, ETTm1, and ETTm2) under multiple forecasting settings, including univariate-to-univariate, multivariate-to-univariate, and multivariate-to-multivariate prediction, with horizons of 96, 192, 336, and 720. In the multivariate-to-multivariate setting, the evolved architectures achieve competitive and, in several cases, improved mean squared error relative to a strong Transformer-based baseline. Additional analyses examine performance differences across forecasting settings and report wall-clock training time to provide a coarse indication of computational cost. Overall, the results demonstrate that evolutionary search can effectively discover flexible and high-performing Transformer-like architectures for multivariate time-series forecasting within practical runtime constraints.

Problem

• time series forecasting
• neural architecture search
• Transformer
• evolutionary algorithm
• multivariate time series

Innovation

• Evolutionary Neural Architecture Search
• Transformer-like Models
• Time Series Forecasting
• Modular Genome Representation
• Structural Repair Mechanism