Fourier Feature Pyramids for Physics-Informed Neural Networks

2026-05-22

AI Summary

This work addresses the limitations of conventional physics-informed neural networks (PINNs) in solving partial differential equations (PDEs), particularly their insufficient accuracy and rapidly degrading computational efficiency as model size increases. The authors propose the beignet architecture, which innovatively replaces random Fourier embeddings with a trainable, bandwidth-controllable multi-resolution Fourier feature pyramid. By integrating Fourier interpolation, automatic differentiation, and fast Fourier transforms (FFT), the method enables efficient derivative computation and stable optimization. This approach substantially improves parameter efficiency, achieving higher solution accuracy with fewer parameters across multiple PDE benchmarks. Notably, on the inviscid Burgers’ equation—a problem exhibiting shock formation—the method attains near-machine-precision residuals when optimized with the Adam optimizer.

Abstract

We present an improved neural field architecture for solving partial differential equations (PDEs). Current physics-informed neural networks (PINNs) provide a flexible framework for solving PDEs, but they struggle to achieve highly accurate solutions and require computation that scales poorly with parameter count. Our model, which we call beignet (Bandlimited Embedding with Interpolated Grid Network), replaces the random Fourier feature embedding used by existing PINN models with a trainable multi-resolution Fourier feature pyramid. To query beignet at a continuous coordinate, we use Fourier interpolation at each level of the pyramid to return features at the input coordinate, and then decode this vector with a fully-connected neural network trunk. Our model provides multiple benefits: 1) Spatial derivatives can be computed efficiently by using the chain rule to compose derivatives of the neural network computed with automatic differentiation with derivatives of the feature grid computed spectrally by the Fast Fourier transform (FFT). 2) beignet can achieve higher accuracy in a compute-efficient manner by scaling the parameter count of this Fourier feature pyramid, instead of the less-efficient strategy of scaling the neural network architecture. 3) beignet can directly control the representation bandlimit, resulting in more stable optimization for difficult PDEs. We demonstrate that beignet finds significantly more accurate solutions on PDE benchmarks using fewer parameters than state-of-the-art PINN methods. We further evaluate beignet on the self-similar inviscid Burgers blowup problem and show that it can minimize residuals to near machine precision using Adam, an accuracy regime previously attained only by using computationally expensive higher-order optimizers.

Problem

• Physics-Informed Neural Networks
• Partial Differential Equations
• Accuracy
• Computational Efficiency
• Fourier Features

Innovation

• Fourier feature pyramid
• physics-informed neural networks
• spectral derivatives
• bandlimited embedding
• efficient PDE solving