Manifold meta-learning for reduced-complexity neural system identification

2025-04-16

AI Summary

To address the high training/inference overhead and poor generalization of over-parameterized deep learning models in nonlinear dynamical system identification under few-shot settings, this paper proposes a manifold-based meta-learning framework. The method identifies a low-dimensional, generalizable manifold within the parameter space of over-parameterized neural networks and introduces, for the first time, an auxiliary neural network that directly maps input datasets onto this manifold—bypassing costly second-order gradient computations required in conventional meta-learning. Evaluated on modeling the Bouc–Wen oscillator family, the framework significantly improves modeling accuracy in data-scarce regimes while reducing both meta-training and inference complexity. Key contributions include: (1) tight integration of parameter-space manifold modeling with meta-learning; (2) an efficient, second-order-free mapping mechanism; and (3) a lightweight, high-accuracy identification paradigm tailored for physics-informed systems.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Metareasoning and Metaheuristics
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

System identification has greatly benefited from deep learning techniques, particularly for modeling complex, nonlinear dynamical systems with partially unknown physics where traditional approaches may not be feasible. However, deep learning models often require large datasets and significant computational resources at training and inference due to their high-dimensional parameterizations. To address this challenge, we propose a meta-learning framework that discovers a low-dimensional manifold within the parameter space of an over-parameterized neural network architecture. This manifold is learned from a meta-dataset of input-output sequences generated by a class of related dynamical systems, enabling efficient model training while preserving the network's expressive power for the considered system class. Unlike bilevel meta-learning approaches, our method employs an auxiliary neural network to map datasets directly onto the learned manifold, eliminating the need for costly second-order gradient computations during meta-training and reducing the number of first-order updates required in inference, which could be expensive for large models. We validate our approach on a family of Bouc-Wen oscillators, which is a well-studied nonlinear system identification benchmark. We demonstrate that we are able to learn accurate models even in small-data scenarios.

Problem

• Reduces complexity in neural system identification
• Enables efficient training with small datasets
• Eliminates costly second-order gradient computations

Innovation

• Meta-learning discovers low-dimensional neural network manifold
• Auxiliary network maps datasets to manifold efficiently
• Reduces gradient computations and first-order updates