Relating Piecewise Linear Kolmogorov Arnold Networks to ReLU Networks

2025-03-03

AI Summary

Establishing a rigorous equivalence between piecewise-linear Kolmogorov–Arnold networks (KANs) and ReLU networks remains an open theoretical challenge, hindering integration of KANs into established deep learning frameworks. Method: We propose the first fully explicit, bidirectional, lossless conversion algorithm via constructive function approximation and structural mapping—enabling exact translation of any piecewise-linear KAN into a ReLU network with controlled depth and parameter bounds, and vice versa. Contribution/Results: We prove that piecewise-linear KANs and ReLU networks are fundamentally equivalent in expressive power, thereby bridging a critical theoretical gap between the emerging KAN paradigm and classical neural network theory. This equivalence not only reveals their intrinsic unification but also enables principled interpretability analysis of KANs, facilitates hardware-aware deployment, and supports seamless interoperability with standard deep learning libraries—without approximation or architectural constraints.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Kolmogorov-Arnold Networks are a new family of neural network architectures which holds promise for overcoming the curse of dimensionality and has interpretability benefits (arXiv:2404.19756). In this paper, we explore the connection between Kolmogorov Arnold Networks (KANs) with piecewise linear (univariate real) functions and ReLU networks. We provide completely explicit constructions to convert a piecewise linear KAN into a ReLU network and vice versa.

Problem

• Connects Kolmogorov-Arnold Networks to ReLU networks
• Explores piecewise linear functions in neural networks
• Provides explicit conversions between KANs and ReLU networks

Innovation

• Connects Kolmogorov Arnold Networks to ReLU networks
• Converts piecewise linear KANs into ReLU networks
• Enables bidirectional conversion between KANs and ReLU networks