Discretization Error of Fourier Neural Operators

2024-05-03

arXiv.org

AI Summary

This work addresses aliasing errors introduced in the discrete implementation of Fourier Neural Operators (FNOs), which remain theoretically unquantified despite their practical significance. Specifically, the discrepancy between the continuous FNO formulation and its grid-based discretization has not been systematically characterized, and isolating this discretization error from other sources—such as approximation and optimization errors—remains challenging. Method: Leveraging tools from Fourier analysis, numerical functional analysis, and FFT theory, we derive an explicit algebraic convergence rate for the discretization error with respect to grid resolution and establish its quantitative dependence on the Sobolev regularity of the input function. Contribution/Results: We provide the first verifiable upper bound on this error, revealing intrinsic trade-offs among resolution, input smoothness, and model stability. Numerical experiments confirm the theoretical prediction: for inputs with higher Sobolev regularity, the error decays significantly under mesh refinement—thereby furnishing a rigorous foundation for reliable discrete FNO design.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Other Foundations of Machine Learning
• Computer Vision: Other Foundations of Computer Vision

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Operator learning is a variant of machine learning that is designed to approximate maps between function spaces from data. The Fourier Neural Operator (FNO) is a common model architecture used for operator learning. The FNO combines pointwise linear and nonlinear operations in physical space with pointwise linear operations in Fourier space, leading to a parameterized map acting between function spaces. Although FNOs formally involve convolutions of functions on a continuum, in practice the computations are performed on a discretized grid, allowing efficient implementation via the FFT. In this paper, the aliasing error that results from such a discretization is quantified and algebraic rates of convergence in terms of the grid resolution are obtained as a function of the regularity of the input. Numerical experiments that validate the theory and describe model stability are performed.

Problem

• Analyzing discretization error between continuous and implemented Fourier Neural Operators
• Establishing algebraic convergence rates based on grid resolution and input regularity
• Developing algorithm to optimize training using error decomposition analysis

Innovation

• FNO combines physical and Fourier space operations
• Analyzes discretization error versus continuum definition
• Leverages error decomposition to optimize training time