🤖 AI Summary
To address the poor stability, high computational cost, and low accuracy of ODE solvers in training physics-enhanced neural ordinary differential equations (PeNODEs), this work reformulates training as a dynamic optimization problem. It employs a high-order implicit Runge–Kutta method—based on flipped Legendre–Gauss–Radau collocation points—for temporal discretization, yielding a large-scale nonlinear programming (NLP) problem solved efficiently via Ipopt. Our approach achieves, for the first time, joint optimization of neural parameters and state trajectories; extends direct collocation to incorporate physics-based constraints; and releases an open-source implementation supporting parallel computation and interoperability with OpenModelica, while also accommodating Neural DAE training. Experiments on the Quarter Vehicle and Van der Pol systems demonstrate substantial improvements in accuracy and training speed, superior generalization with smaller networks, and enhanced numerical robustness—particularly for stiff systems.
📝 Abstract
We propose a novel approach for training Physics-enhanced Neural ODEs (PeNODEs) by expressing the training process as a dynamic optimization problem. The full model, including neural components, is discretized using a high-order implicit Runge-Kutta method with flipped Legendre-Gauss-Radau points, resulting in a large-scale nonlinear program (NLP) efficiently solved by state-of-the-art NLP solvers such as Ipopt. This formulation enables simultaneous optimization of network parameters and state trajectories, addressing key limitations of ODE solver-based training in terms of stability, runtime, and accuracy. Extending on a recent direct collocation-based method for Neural ODEs, we generalize to PeNODEs, incorporate physical constraints, and present a custom, parallelized, open-source implementation. Benchmarks on a Quarter Vehicle Model and a Van-der-Pol oscillator demonstrate superior accuracy, speed, and generalization with smaller networks compared to other training techniques. We also outline a planned integration into OpenModelica to enable accessible training of Neural DAEs.