UKAN: Unbound Kolmogorov-Arnold Network Accompanied with Accelerated Library

2024-08-20

arXiv.org

AI Summary

Traditional Kolmogorov–Arnold networks (KANs) rely on predefined bounded B-spline grids, limiting their ability to approximate functions over unbounded domains. This work proposes Unbounded Kolmogorov–Arnold Networks (UKAN), the first KAN variant that eliminates fixed B-spline grids and explicit parameter storage. Instead, UKAN employs an infinite symmetric grid, positional encoding, and a lightweight coefficient generator (CG) to dynamically synthesize basis functions—obviating input normalization or manual domain boundary specification. A dedicated GPU-accelerated library enables efficient, vectorized B-spline evaluation. Experiments demonstrate UKAN’s superior performance across regression, classification, and generative tasks. Crucially, its computational complexity reduces to *O*(1) per activation—contrasting sharply with traditional KAN’s *O*(grid size)—while achieving substantial gains in memory efficiency and throughput. UKAN establishes a scalable, domain-agnostic paradigm for unbounded function approximation.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

In this work, we present a GPU-accelerated library for the underlying components of Kolmogorov-Arnold Networks (KANs), along with an algorithm to eliminate bounded grids in KANs. The GPU-accelerated library reduces the computational complexity of Basis Spline (B-spline) evaluation by a factor of $mathcal{O}$(grid size) compared to existing codes, enabling batch computation for large-scale learning. To overcome the limitations of traditional KANs, we introduce Unbounded KANs (UKANs), which eliminate the need for a bounded grid and a fixed number of B-spline coefficients. To do so, we replace the KAN parameters (B-spline coefficients) with a coefficient generator (CG) model. The inputs to the CG model are designed based on the idea of an infinite symmetric grid extending from negative infinity to positive infinity. The positional encoding of grid group, a sequential collection of B-spline grid indexes, is fed into the CG model, and coefficients are consumed by the efficient implementation (matrix representations) of B-spline functions to generate outputs. We perform several experiments on regression, classification, and generative tasks, which are promising. In particular, UKAN does not require data normalization or a bounded domain for evaluation. Additionally, our benchmarking results indicate the superior memory and computational efficiency of our library compared to existing codes.

Problem

• UKANs enable function approximation on unbounded domains without data normalization
• The method reduces computational costs through GPU acceleration and memory optimization
• It achieves superior efficiency in regression, classification, and molecular prediction tasks

Innovation

• UKANs use coefficient-generator model for unbounded grids
• GPU-accelerated library reduces B-spline evaluation complexity
• Couples MLPs with KANs using positional encoding