Nyström-Accelerated Primal LS-SVMs: Breaking the $O(an^3)$ Complexity Bottleneck for Scalable ODEs Learning

2025-10-05

AI Summary

To address the high computational complexity—O(an³), where a=1 for linear and a=27 for nonlinear ODEs—of kernel-based methods (e.g., LS-SVM) in solving ordinary differential equations (ODEs), this paper proposes a Nyström-accelerated primal-space LS-SVM framework. The core method constructs, for the first time, a one-dimensional temporal domain-to-m-dimensional explicit feature-space Nyström mapping and its analytical derivatives, enabling direct embedding of differential constraints into the primal space. This reduces computational complexity from O(n³) to O(m³), with m ≪ n. The approach achieves both high accuracy and scalability: on 16 benchmark ODEs, it accelerates computation by 10–6,000× over classical LS-SVM and physics-informed neural networks (PINNs), attains errors <0.13%, improves RMSE by up to 72%, and supports solutions with tens of thousands of time steps.

Technology Category

• Machine Learning: Kernel Methods
• Search and Optimization: Non-convex Optimization
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

A major problem of kernel-based methods (e.g., least squares support vector machines, LS-SVMs) for solving linear/nonlinear ordinary differential equations (ODEs) is the prohibitive $O(an^3)$ ($a=1$ for linear ODEs and 27 for nonlinear ODEs) part of their computational complexity with increasing temporal discretization points $n$. We propose a novel Nyström-accelerated LS-SVMs framework that breaks this bottleneck by reformulating ODEs as primal-space constraints. Specifically, we derive for the first time an explicit Nyström-based mapping and its derivatives from one-dimensional temporal discretization points to a higher $m$-dimensional feature space ($1< mle n$), enabling the learning process to solve linear/nonlinear equation systems with $m$-dependent complexity. Numerical experiments on sixteen benchmark ODEs demonstrate: 1) $10-6000$ times faster computation than classical LS-SVMs and physics-informed neural networks (PINNs), 2) comparable accuracy to LS-SVMs ($<0.13%$ relative MAE, RMSE, and $left | y-hat{y} ight | _{infty } $difference) while maximum surpassing PINNs by 72% in RMSE, and 3) scalability to $n=10^4$ time steps with $m=50$ features. This work establishes a new paradigm for efficient kernel-based ODEs learning without significantly sacrificing the accuracy of the solution.

Problem

• Reducing computational complexity of kernel-based ODE solvers
• Breaking scalability bottleneck in LS-SVMs for differential equations
• Accelerating ODE learning while maintaining solution accuracy

Innovation

• Nyström-accelerated LS-SVMs framework for ODEs
• Explicit Nyström mapping to higher feature space
• Reformulates ODEs as primal-space constraints