Sparse Orthogonal Regression Technique: A Spectral Framework for Equation Discovery, Approximation, and Integration

2026-08-13

AI Summary

This work addresses the challenges of discovering differential equations, approximating functions, and estimating high-dimensional integrals from noisy, non-uniformly sampled data by introducing Sparse Orthogonal Regression Technique (SORT). The method reformulates equation discovery as a spectral coefficient learning problem, directly inferring coefficients of an orthogonal basis expansion from observational data via L1 regularization—without requiring a predefined symbolic library, explicit numerical integration, or inner product evaluations. Its core innovation lies in treating basis function design as the central modeling choice, enabling consistent model order scaling and multi-task reusability. Experiments demonstrate that when the basis functions align with the underlying problem structure, SORT matches or outperforms existing approaches under sparse sampling, noisy derivatives, and representation mismatch, while low-order dominant coefficients remain stable as model complexity increases.

Abstract

We develop the Sparse Orthogonal Regression Technique (SORT), a sparse spectral framework for learning orthonormal-basis expansions from noisy and irregularly sampled data. SORT estimates expansion coefficients directly from observations using L1-regularized regression, avoiding explicit quadrature or analytic inner-product evaluation. The central application is data-driven discovery of ordinary differential equations: vector fields are represented in chosen orthogonal bases and learned as sparse coefficient expansions. This provides a complementary route to symbolic regression, grammar-based discovery, and SINDy-style sparse identification by first recovering a compact spectral representation, which can later guide searches for simpler analytic forms. Across the dynamical-system experiments, SORT matches or improves upon library-based sparse-regression baselines when the basis is well adapted to the problem, and shows more stable degradation under sparse sampling, noisy derivative estimates, and representation mismatch. Specific examples illustrate why this representation is useful: if a finite library misses the problem-specific nonlinearity, the resulting model can fail. SORT is not immune to mismatch, but it shifts the problem away from brittle selection among generic terms to basis design adapted to the problem domain. The experiments also show that dominant low-order coefficients persist as model order increases, supporting order-consistent model growth. Beyond equation discovery, the same learned expansion supports nonlinear approximation and estimation of complex, high-dimensional integrals by coefficient readout. Overall, SORT provides a reusable intermediate representation for system identification, approximation, and integration, while making basis design an explicit part of the scientific modeling problem.

Problem

• equation discovery
• sparse regression
• orthogonal basis
• noisy data
• irregular sampling

Innovation

• Sparse Orthogonal Regression
• Spectral Representation
• Equation Discovery
• L1-Regularized Regression
• Orthonormal Basis Expansion