Kernel Neural Operators (KNOs) for Scalable, Memory-efficient, Geometrically-flexible Operator Learning

2024-06-30

arXiv.org

AI Summary

Existing operator learning methods for irregular geometric domains suffer from high memory consumption and poor geometric adaptability. Method: This paper proposes the Deep Integral Operator (DIO) framework, which employs learnable compactly supported kernel functions and a sparsity-aware parametrization scheme to jointly ensure smoothness and computational efficiency; it further incorporates adaptive numerical quadrature to achieve geometry-agnostic modeling, eliminating reliance on structured grids. Contribution/Results: On standard benchmarks, DIO achieves higher accuracy than state-of-the-art neural operators, with improved training and test accuracy. It reduces trainable parameters by over an order of magnitude, significantly enhancing memory efficiency, generalization capability, and geometric robustness.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

This paper introduces the Kernel Neural Operator (KNO), a novel operator learning technique that uses deep kernel-based integral operators in conjunction with quadrature for function-space approximation of operators (maps from functions to functions). KNOs use parameterized, closed-form, finitely-smooth, and compactly-supported kernels with trainable sparsity parameters within the integral operators to significantly reduce the number of parameters that must be learned relative to existing neural operators. Moreover, the use of quadrature for numerical integration endows the KNO with geometric flexibility that enables operator learning on irregular geometries. Numerical results demonstrate that on existing benchmarks the training and test accuracy of KNOs is higher than popular operator learning techniques while using at least an order of magnitude fewer trainable parameters. KNOs thus represent a new paradigm of low-memory, geometrically-flexible, deep operator learning, while retaining the implementation simplicity and transparency of traditional kernel methods from both scientific computing and machine learning.

Problem

• Developing scalable operator learning with memory-efficient neural architectures
• Enabling geometrically-flexible operator approximation on irregular domains
• Overcoming dimensionality curse through expressive trainable kernel designs

Innovation

• Uses deep kernel-based integral operators for approximation
• Decouples kernel choice from numerical integration scheme
• Leverages dimension-wise factorization for efficiency