Selection, Representation, and Execution in Sparse Fourier Neural Operators

2026-08-30

AI Summary

研究通过区分稀疏表示、存储参数、理论操作数和测量运行时间，探索了稀疏Fourier神经算子的不同实现路径，以解决模型大小和推理成本问题。

Abstract

Sparse representations are often expected to make models smaller and also reduce inference cost. For Fourier Neural Operators (FNOs), these objectives are not equivalent or do not always align: removing parts of the learned operator can leave the underlying transforms and dense computations unchanged, while changing the grid on which the model is evaluated can introduce overhead of its own. We therefore distinguish sparsity in the representation, in the stored parameters, in the theoretical operation count, and in measured runtime, and present an empirical study of several routes toward sparse FNOs that tests each transition between them separately. Coarsening the execution grid reduces the theoretical cost without reducing measured latency, and adding a correction term recovers accuracy at the cost of making the model slower. Even an 83\% parameter reduction remains slower than the dense baseline under ordinary execution. These results motivate a stricter definition of useful sparsity: the deployed operator must preserve solution accuracy and map its reduced support to a genuinely cheaper execution path.

Problem

• Sparse Fourier Neural Operators
• inference cost
• representation sparsity
• execution grid
• parameter reduction

Innovation

• Sparse Fourier Neural Operators
• sparsity in representation
• theoretical operation count
• measured runtime
• useful sparsity