Learning System Dynamics without Forgetting

2024-06-30

arXiv.org

AI Summary

This work addresses the challenge of multi-system trajectory prediction under continuously evolving dynamic patterns. We propose Continual Dynamics Learning (CDL), a novel paradigm that overcomes the limitation of existing methods—namely, their reliance on static, system-specific dynamical modeling. We formally define the CDL problem and introduce Bio-CDL, the first benchmark tailored for continual learning in biological dynamics. To tackle CDL, we design the Mode-switching Graph ODE model, which integrates graph ordinary differential equations, subnet-based incremental learning, and a differentiable mode-switching mechanism to enable cross-system adaptive transfer and long-term memory retention. Extensive experiments on Bio-CDL demonstrate substantial improvements over state-of-the-art baselines, validating both effective dynamics transfer and stable long-horizon prediction. Our implementation is publicly available.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Multiagent Systems: Multiagent Learning
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Observation-based trajectory prediction for systems with unknown dynamics is essential in fields such as physics and biology. Most existing approaches are limited to learning within a single system with fixed dynamics patterns. However, many real-world applications require learning across systems with evolving dynamics patterns, a challenge that has been largely overlooked. To address this, we systematically investigate the problem of Continual Dynamics Learning (CDL), examining task configurations and evaluating the applicability of existing techniques, while identifying key challenges. In response, we propose the Mode-switching Graph ODE (MS-GODE) model, which integrates the strengths LG-ODE and sub-network learning with a mode-switching module, enabling efficient learning over varying dynamics. Moreover, we construct a novel benchmark of biological dynamic systems for CDL, Bio-CDL, featuring diverse systems with disparate dynamics and significantly enriching the research field of machine learning for dynamic systems. Our code available at https://github.com/QueuQ/MS-GODE.

Problem

• Learning evolving system dynamics
• Overcoming single-system learning limits
• Proposing Mode-switching Graph ODE model

Innovation

• Mode-switching Graph ODE model
• Continual Dynamics Learning approach
• Bio-CDL benchmark for dynamics