Adaptive Nonlinear Vector Autoregression: Robust Forecasting for Noisy Chaotic Time Series

2025-07-11

AI Summary

Traditional nonlinear vector autoregression (NVAR) and reservoir computing (RC) suffer from poor noise robustness and limited scalability to high dimensions—due to fixed nonlinear mappings and reliance on matrix inversion—when applied to chaotic time-series forecasting. To address these limitations, this paper proposes the Adaptive Nonlinear Vector Autoregression (ANVAR) model. ANVAR integrates delay-coordinate embedding with a learnable shallow multilayer perceptron (MLP), enabling end-to-end joint optimization of the nonlinear feature mapping and the linear readout layer—thereby eliminating handcrafted nonlinearities and grid-search-based hyperparameter tuning. On benchmark tasks including the Lorenz-63 and ENSO systems, ANVAR consistently outperforms standard NVAR under both noise-free conditions and additive Gaussian noise (SNR ≤ 10 dB). Moreover, it maintains robust predictive performance even at low sampling rates. The model achieves superior accuracy, enhanced noise resilience, and improved scalability to higher-dimensional settings.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Learning & Optimization for NLP
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Nonlinear vector autoregression (NVAR) and reservoir computing (RC) have shown promise in forecasting chaotic dynamical systems, such as the Lorenz-63 model and El Nino-Southern Oscillation. However, their reliance on fixed nonlinearities - polynomial expansions in NVAR or random feature maps in RC - limits their adaptability to high noise or real-world data. These methods also scale poorly in high-dimensional settings due to costly matrix inversion during readout computation. We propose an adaptive NVAR model that combines delay-embedded linear inputs with features generated by a shallow, learnable multi-layer perceptron (MLP). The MLP and linear readout are jointly trained using gradient-based optimization, enabling the model to learn data-driven nonlinearities while preserving a simple readout structure. Unlike standard NVAR, our approach avoids the need for an exhaustive and sensitive grid search over ridge and delay parameters. Instead, tuning is restricted to neural network hyperparameters, improving scalability. Initial experiments on chaotic systems tested under noise-free and synthetically noisy conditions showed that the adaptive model outperformed the standard NVAR in predictive accuracy and showed robust forecasting under noisy conditions with a lower observation frequency.

Problem

• Improving forecasting accuracy for noisy chaotic time series
• Reducing reliance on fixed nonlinearities in NVAR and RC
• Enhancing scalability by avoiding costly matrix inversion

Innovation

• Combines delay-embedded inputs with learnable MLP
• Jointly trains MLP and linear readout via gradients
• Eliminates grid search for ridge and delay parameters