Upper Generalization Bounds for Neural Oscillators

2026-03-10

AI Summary

This work addresses the lack of theoretical characterization of the generalization capability of neural oscillators based on second-order ordinary differential equations (ODEs). Focusing on neural oscillators composed of a second-order ODE coupled with a multilayer perceptron (MLP), we derive, for the first time, a generalization upper bound within the Rademacher complexity and PAC learning frameworks for approximating causal, uniformly continuous, and asymptotically incrementally stable dynamical systems. Our analysis shows that the approximation error grows polynomially with both the MLP size and the time horizon. Furthermore, enforcing Lipschitz regularization and constraining the MLP weight norms effectively enhances generalization. Empirical validation on seismic response modeling of the Bouc–Wen nonlinear system confirms the predicted power-law relationship between error and model complexity, and demonstrates that norm constraints significantly improve modeling accuracy under limited data regimes.

Technology Category

• Machine Learning: Other Foundations of Machine Learning
• Natural Language Processing: Learning & Optimization for NLP
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Neural oscillators that originate from the second-order ordinary differential equations (ODEs) have shown competitive performance in learning mappings between dynamic loads and responses of complex nonlinear structural systems. Despite this empirical success, theoretically quantifying the generalization capacities of their neural network architectures remains undeveloped. In this study, the neural oscillator consisting of a second-order ODE followed by a multilayer perceptron (MLP) is considered. Its upper probably approximately correct (PAC) generalization bound for approximating causal and uniformly continuous operators between continuous temporal function spaces and that for approximating the uniformly asymptotically incrementally stable second-order dynamical systems are derived by leveraging the Rademacher complexity framework. The theoretical results show that the estimation errors grow polynomially with respect to both the MLP size and the time length, thereby avoiding the curse of parametric complexity. Furthermore, the derived error bounds demonstrate that constraining the Lipschitz constants of the MLPs via loss function regularization can improve the generalization ability of the neural oscillator. A numerical study considering a Bouc-Wen nonlinear system under stochastic seismic excitation validates the theoretically predicted power laws of the estimation errors with respect to the sample size and time length, and confirms the effectiveness of constraining MLPs' matrix and vector norms in enhancing the performance of the neural oscillator under limited training data.

Problem

• neural oscillators
• generalization bounds
• Rademacher complexity
• nonlinear dynamical systems
• PAC learning

Innovation

• neural oscillators
• generalization bounds
• Rademacher complexity
• Lipschitz regularization
• second-order ODEs