Global universal approximation of functional input maps on weighted spaces

2023-06-05

arXiv.org

AI Summary

Global approximation of temporal functions—such as path functionals—in large-scale or infinite-dimensional weighted spaces remains challenging, particularly due to the limitations of conventional compact-support assumptions. Method: We propose functional neural networks tailored for weighted spaces, featuring additive mappings, scalar-valued activations, and linear readouts. This architecture circumvents traditional compact-support constraints and unifies the modeling of signature-based linear functionals and non-anticipative path functionals. We further establish an equivalence between the reproducing kernel Hilbert space (RKHS) induced by the signature kernel and the Cameron–Martin space of associated Gaussian processes. Contributions: First, we prove a universal approximation theorem: additive networks globally approximate continuous functions on weighted spaces. Second, we demonstrate that signature-based linear functionals are universally approximable within this framework. Third, we provide a Gaussian-process-based theoretical foundation for uncertainty quantification in signature kernel regression.

Abstract

We introduce so-called functional input neural networks defined on a possibly infinite dimensional weighted space with values also in a possibly infinite dimensional output space. To this end, we use an additive family to map the input weighted space to the hidden layer, on which a non-linear scalar activation function is applied to each neuron, and finally return the output via some linear readouts. Relying on Stone-Weierstrass theorems on weighted spaces, we can prove a global universal approximation result on weighted spaces for continuous functions going beyond the usual approximation on compact sets. This then applies in particular to approximation of (non-anticipative) path space functionals via functional input neural networks. As a further application of the weighted Stone-Weierstrass theorem we prove a global universal approximation result for linear functions of the signature. We also introduce the viewpoint of Gaussian process regression in this setting and emphasize that the reproducing kernel Hilbert space of the signature kernels are Cameron-Martin spaces of certain Gaussian processes. This paves a way towards uncertainty quantification for signature kernel regression.

Problem

• High-dimensional Space Approximation
• Time Series Function Approximation
• Predictive Uncertainty

Innovation

• Neural Network Architecture
• Signature Kernels
• Gaussian Process Regression