How deep is your network? Deep vs. shallow learning of transfer operators

2025-09-24

AI Summary

Deep learning approaches for learning transfer operators and their spectral decompositions suffer from hyperparameter sensitivity, slow convergence, and poor interpretability. To address these issues, we propose RaNNDy—a shallow learning framework based on random neural networks. RaNNDy fixes the hidden-layer weights and solves the output layer in closed form, directly yielding eigenfunctions of the Koopman, Perron–Frobenius, or Schrödinger operators. It further incorporates ensemble learning to quantify uncertainty in spectral estimates. Compared to conventional deep learning, RaNNDy drastically reduces training cost while enhancing robustness and interpretability. We evaluate RaNNDy on stochastic dynamical systems, protein folding, and the quantum harmonic oscillator, demonstrating both high accuracy and computational efficiency. These results validate the effectiveness and promise of shallow random networks for spectral analysis of complex systems.

Technology Category

• Machine Learning: Learning with Manifolds
• Reasoning under Uncertainty: Stochastic Optimization
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Social networks and social learning
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We propose a randomized neural network approach called RaNNDy for learning transfer operators and their spectral decompositions from data. The weights of the hidden layers of the neural network are randomly selected and only the output layer is trained. The main advantage is that without a noticeable reduction in accuracy, this approach significantly reduces the training time and resources while avoiding common problems associated with deep learning such as sensitivity to hyperparameters and slow convergence. Additionally, the proposed framework allows us to compute a closed-form solution for the output layer which directly represents the eigenfunctions of the operator. Moreover, it is possible to estimate uncertainties associated with the computed spectral properties via ensemble learning. We present results for different dynamical operators, including Koopman and Perron-Frobenius operators, which have important applications in analyzing the behavior of complex dynamical systems, and the Schrödinger operator. The numerical examples, which highlight the strengths but also weaknesses of the proposed framework, include several stochastic dynamical systems, protein folding processes, and the quantum harmonic oscillator.

Problem

• Learning transfer operators and spectral decompositions from data
• Reducing training time and avoiding deep learning issues
• Computing eigenfunctions and estimating spectral uncertainties

Innovation

• Randomized neural network for learning transfer operators
• Training only output layer with random hidden weights
• Closed-form solution for eigenfunctions and uncertainty estimation