Higher-Order Fourier Neural Operator: Explicit Mode Mixer for Nonlinear PDEs

2026-06-26

AI Summary

This work addresses the limitation of existing Fourier Neural Operators (FNOs) in effectively modeling structured couplings among Fourier modes in nonlinear partial differential equations. To overcome this, the authors propose a Higher-Order Spectral Convolution mechanism (HO-FNO), which explicitly introduces multilinear mode mixing in the frequency domain. This approach generalizes FNO’s diagonal modulation to higher-order nonlinear interactions, embedding inductive biases aligned with the dynamics of nonlinear PDEs. Empirical results demonstrate that HO-FNO significantly outperforms current spectral neural operators across multiple benchmarks: a single-layer HO-FNO surpasses a 16-layer FNO in highly nonlinear regimes and matches or exceeds the performance of advanced Transformers and state-space models.

Abstract

Neural operators provide deep neural networks for learning mappings between function spaces. Among them, the Fourier Neural Operator (FNO) is particularly effective: its spectral convolution relies on low-dimensional Fourier-domain representations and can handle inputs at different resolutions. This design aligns well with settings where the Fourier basis diagonalizes the underlying operator, such as linear, constant-coefficient PDEs on periodic domains, in which Fourier modes evolve independently. However, nonlinear PDEs may benefit from an additional inductive bias, as they exhibit structured interactions between modes, governed by polynomial nonlinearities. To capture this inductive bias, we introduce the Higher-Order Spectral Convolution, a spectral mixer that extends FNO from diagonal modulation to explicit n-linear mode mixing, aligned with the dynamics of nonlinear PDEs. Our experiments on standard benchmarks show that the proposed Higher-Order FNO (HO-FNO) retains the efficiency of FNO-based architectures and consistently improves over other spectral neural operators. HO-FNO also performs on par with or better than state-of-the-art transformers and state-space models on several datasets, with stronger gains in highly nonlinear regimes, such as the Poisson equation with polynomial forcing, where a single HO-FNO layer outperforms FNO models with up to 16 layers. We open-source our code for reproducibility at: https://github.com/AlexColagrande/HO-FNO.

Problem

• nonlinear PDEs
• Fourier Neural Operator
• mode interactions
• spectral convolution
• inductive bias

Innovation

• Higher-Order Spectral Convolution
• Fourier Neural Operator
• Nonlinear PDEs
• Mode Mixing
• Neural Operators