🤖 AI Summary
Conventional Convolutional Conditional Neural Processes (ConvCNPs) struggle with long-range dependencies in few-shot, irregularly sampled function modeling due to their reliance on local spatial convolutions; increasing kernel size incurs prohibitive computational overhead. Method: This work introduces SpectralCNP—the first integration of Fourier Neural Operator principles into the CNP framework—replacing local spatial convolutions with global spectral-domain convolutions via parameterized Fourier transforms. This enables robust modeling of sparse, unstructured observations. Contribution/Results: Trained via maximum likelihood, SpectralCNP achieves significant improvements over ConvCNP across multiple meta-learning and functional regression benchmarks. Notably, it delivers superior predictive accuracy and calibration—particularly under low-data regimes and irregular sampling—while maintaining computational efficiency through spectral parameterization.
📝 Abstract
Conditional Neural Processes (CNPs) constitute a family of probabilistic models that harness the flexibility of neural networks to parameterize stochastic processes. Their capability to furnish well-calibrated predictions, combined with simple maximum-likelihood training, has established them as appealing solutions for addressing various learning problems, with a particular emphasis on meta-learning. A prominent member of this family, Convolutional Conditional Neural Processes (ConvCNPs), utilizes convolution to explicitly introduce translation equivariance as an inductive bias. However, ConvCNP's reliance on local discrete kernels in its convolution layers can pose challenges in capturing long-range dependencies and complex patterns within the data, especially when dealing with limited and irregularly sampled observations from a new task. Building on the successes of Fourier neural operators (FNOs) for approximating the solution operators of parametric partial differential equations (PDEs), we propose Spectral Convolutional Conditional Neural Processes (SConvCNPs), a new addition to the NPs family that allows for more efficient representation of functions in the frequency domain.