🤖 AI Summary
This work investigates the universal approximation capabilities of Transformers and neural integral operators in Banach spaces. Addressing three core problems: (1) whether Transformers can universally approximate integral operators between Hölder spaces; (2) whether neural integral operators exist that universally approximate arbitrary continuous linear or nonlinear operators between Banach spaces; and (3) how to overcome regularity constraints for broader approximation. We first establish, for the first time, the universal approximation property of Transformers for integral operators acting between Hölder spaces. Second, we propose a generalized neural integral operator based on the Gavurin integral and prove its universal approximation theorem for continuous operators between arbitrary Banach spaces. Third, we incorporate Leray–Schauder mappings into the Transformer architecture to eliminate dependence on smoothness assumptions on input/output spaces. These results provide a rigorous functional-analytic foundation for operator learning and extend the theoretical scope of deep learning in infinite-dimensional tasks such as PDE solving and physics-informed modeling.
📝 Abstract
We study the universal approximation properties of transformers and neural integral operators for operators in Banach spaces. In particular, we show that the transformer architecture is a universal approximator of integral operators between H""older spaces. Moreover, we show that a generalized version of neural integral operators, based on the Gavurin integral, are universal approximators of arbitrary operators between Banach spaces. Lastly, we show that a modified version of transformer, which uses Leray-Schauder mappings, is a universal approximator of operators between arbitrary Banach spaces.