FC-KAN: Function Combinations in Kolmogorov-Arnold Networks

2024-09-03

arXiv.org

AI Summary

To address the limited expressive capacity of Kolmogorov–Arnold Networks (KANs), this paper proposes FC-KAN—a novel KAN architecture that explicitly integrates elementary mathematical functions—including B-splines, Difference-of-Gaussians (DoG), wavelets, radial basis functions (RBFs), and polynomials—via low-dimensional, element-wise operations. We introduce a flexible multi-function composition mechanism encompassing summation, multiplication, quadratic/cubic representations, concatenation, and linear projection. Notably, we present two novel variants: DoG-B-spline and quadratic-linear hybrid functions—the first such incorporation in KAN literature. Extensive evaluation on MNIST and Fashion-MNIST across five independent trials demonstrates that FC-KAN achieves significantly higher average accuracy than standard MLPs and state-of-the-art KAN variants, including BSRBF-KAN, EfficientKAN, and FastKAN. These results empirically validate that explicit, mathematically grounded function composition enhances both the interpretability and modeling capability of neural networks.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

In this paper, we introduce FC-KAN, a Kolmogorov-Arnold Network (KAN) that leverages combinations of popular mathematical functions such as B-splines, wavelets, and radial basis functions on low-dimensional data through element-wise operations. We explore several methods for combining the outputs of these functions, including sum, element-wise product, the addition of sum and element-wise product, representations of quadratic and cubic functions, concatenation, linear transformation of the concatenated output, and others. In our experiments, we compare FC-KAN with a multi-layer perceptron network (MLP) and other existing KANs, such as BSRBF-KAN, EfficientKAN, FastKAN, and FasterKAN, on the MNIST and Fashion-MNIST datasets. Two variants of FC-KAN, which use a combination of outputs from B-splines and Difference of Gaussians (DoG) and from B-splines and linear transformations in the form of a quadratic function, outperformed overall other models on the average of 5 independent training runs. We expect that FC-KAN can leverage function combinations to design future KANs. Our repository is publicly available at: https://github.com/hoangthangta/FC_KAN.

Problem

• K-A Network
• Mathematical Functions
• Performance Improvement

Innovation

• FC-KAN
• B-spline Integration
• Performance Enhancement