FUTON: Fourier Tensor Network for Implicit Neural Representations

2026-02-13

AI Summary

This work addresses the limitations of conventional MLP-based implicit neural representations—namely slow convergence, susceptibility to overfitting noise, and poor extrapolation—by proposing a signal modeling approach grounded in generalized Fourier series. The method parameterizes spectral coefficients via low-rank tensor decomposition and constructs the representation through weighted combinations of orthogonal, separable basis functions. By integrating the smooth periodicity of Fourier bases with the inductive bias of low-rank spectral structures, it achieves universal approximation capability while maintaining linear inference complexity. Experiments demonstrate that the proposed approach outperforms state-of-the-art MLP methods in image and volumetric data representation, achieving 2–5× faster training, and exhibits superior generalization and accelerated convergence in inverse problems such as denoising and super-resolution.

Technology Category

• Machine Learning: Representation Learning
• Computer Vision: Representation Learning for Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Implicit neural representations (INRs) have emerged as powerful tools for encoding signals, yet dominant MLP-based designs often suffer from slow convergence, overfitting to noise, and poor extrapolation. We introduce FUTON (Fourier Tensor Network), which models signals as generalized Fourier series whose coefficients are parameterized by a low-rank tensor decomposition. FUTON implicitly expresses signals as weighted combinations of orthonormal, separable basis functions, combining complementary inductive biases: Fourier bases capture smoothness and periodicity, while the low-rank parameterization enforces low-dimensional spectral structure. We provide theoretical guarantees through a universal approximation theorem and derive an inference algorithm with complexity linear in the spectral resolution and the input dimension. On image and volume representation, FUTON consistently outperforms state-of-the-art MLP-based INRs while training 2--5$\times$ faster. On inverse problems such as image denoising and super-resolution, FUTON generalizes better and converges faster.

Problem

• Implicit Neural Representations
• slow convergence
• overfitting
• poor extrapolation

Innovation

• Fourier Tensor Network
• Implicit Neural Representations
• Low-rank Tensor Decomposition
• Generalized Fourier Series
• Inductive Bias