An Efficient Continual Learning Framework for Multivariate Time Series Prediction Tasks with Application to Vehicle State Estimation

2025-03-03

AI Summary

To address catastrophic forgetting in continual learning of multivariate time series—particularly in dynamic scenarios such as vehicle state estimation—this paper proposes EM-ReSeleCT. The method introduces a novel collaborative selection mechanism for representative multivariate samples, designs a sequence-to-sequence Transformer architecture tailored for multivariate output prediction, and integrates conformal prediction to robustly quantify memory capacity. It jointly incorporates memory replay, autoregressive modeling, and an improved optimization strategy. Evaluated on real-world electric Chevrolet Equinox driving data, EM-ReSeleCT achieves significantly higher prediction accuracy and substantially reduced training time compared to state-of-the-art continual learning approaches, while effectively balancing retention of historical knowledge and adaptation efficiency to new tasks.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Search and Optimization: Learning to Search
• Multiagent Systems: Multiagent Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In continual time series analysis using neural networks, catastrophic forgetting (CF) of previously learned models when training on new data domains has always been a significant challenge. This problem is especially challenging in vehicle estimation and control, where new information is sequentially introduced to the model. Unfortunately, existing work on continual learning has not sufficiently addressed the adverse effects of catastrophic forgetting in time series analysis, particularly in multivariate output environments. In this paper, we present EM-ReSeleCT (Efficient Multivariate Representative Selection for Continual Learning in Time Series Tasks), an enhanced approach designed to handle continual learning in multivariate environments. Our approach strategically selects representative subsets from old and historical data and incorporates memory-based continual learning techniques with an improved optimization algorithm to adapt the pre-trained model on new information while preserving previously acquired information. Additionally, we develop a sequence-to-sequence transformer model (autoregressive model) specifically designed for vehicle state estimation. Moreover, we propose an uncertainty quantification framework using conformal prediction to assess the sensitivity of the memory size and to showcase the robustness of the proposed method. Experimental results from tests on an electric Equinox vehicle highlight the superiority of our method in continually learning new information while retaining prior knowledge, outperforming state-of-the-art continual learning methods. Furthermore, EM-ReSeleCT significantly reduces training time, a critical advantage in continual learning applications.

Problem

• Addresses catastrophic forgetting in continual time series learning.
• Develops a method for vehicle state estimation using continual learning.
• Proposes an uncertainty quantification framework for robustness assessment.

Innovation

• Strategic selection of representative data subsets
• Memory-based continual learning with improved optimization
• Sequence-to-sequence transformer for vehicle state estimation