KAA: Kolmogorov-Arnold Attention for Enhancing Attentive Graph Neural Networks

2025-01-23

AI Summary

Existing graph attention mechanisms suffer from limited expressivity of their scoring functions, leading to inaccurate node importance estimation and constraining GNN performance. To address this, we propose the Kolmogorov–Arnold Attention (KAA) module—the first to incorporate Kolmogorov–Arnold Networks (KANs) into graph attention scoring—employing zero-order B-spline parameterization to achieve a favorable trade-off between high expressivity and low parameter count. We introduce the Maximum Ranking Distance (MRD) metric to quantify the upper bound of scoring error and theoretically prove that KAA possesses near-universal function approximation capability under parameter constraints. KAA is plug-and-play and compatible with diverse attentive GNN backbones. Extensive experiments demonstrate its superiority across multiple node-level and graph-level benchmarks, with improvements exceeding 20% in several cases, validating both its effectiveness and generalizability.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web search models and ranking

Abstract

Graph neural networks (GNNs) with attention mechanisms, often referred to as attentive GNNs, have emerged as a prominent paradigm in advanced GNN models in recent years. However, our understanding of the critical process of scoring neighbor nodes remains limited, leading to the underperformance of many existing attentive GNNs. In this paper, we unify the scoring functions of current attentive GNNs and propose Kolmogorov-Arnold Attention (KAA), which integrates the Kolmogorov-Arnold Network (KAN) architecture into the scoring process. KAA enhances the performance of scoring functions across the board and can be applied to nearly all existing attentive GNNs. To compare the expressive power of KAA with other scoring functions, we introduce Maximum Ranking Distance (MRD) to quantitatively estimate their upper bounds in ranking errors for node importance. Our analysis reveals that, under limited parameters and constraints on width and depth, both linear transformation-based and MLP-based scoring functions exhibit finite expressive power. In contrast, our proposed KAA, even with a single-layer KAN parameterized by zero-order B-spline functions, demonstrates nearly infinite expressive power. Extensive experiments on both node-level and graph-level tasks using various backbone models show that KAA-enhanced scoring functions consistently outperform their original counterparts, achieving performance improvements of over 20% in some cases.

Problem

• Attention Mechanism
• Graph Neural Networks
• Node Scoring

Innovation

• KAA
• Graph Neural Networks
• Attention Mechanism