Attention-based graph neural networks: a survey

2026-05-09

AI Summary

This study addresses the lack of a systematic survey on the integration of attention mechanisms with graph neural networks (GNNs), which has hindered a clear understanding of their developmental trajectory. To bridge this gap, the work proposes a novel two-level taxonomic framework that organizes the field both historically and architecturally. At the upper level, it delineates three chronological phases: Graph Recurrent Attention Networks, Graph Attention Networks, and Graph Transformers. The lower level systematically catalogs representative models within each phase and compares their key characteristics. Through comprehensive literature review and taxonomy-based analysis, the paper elucidates the evolutionary pathway of attention in GNNs, clarifies the strengths and limitations of existing approaches, identifies open challenges, and outlines promising future directions. An accompanying open-source repository is provided to foster ongoing community research.

Abstract

Graph neural networks (GNNs) aim to learn well-trained representations in a lower-dimension space for downstream tasks while preserving the topological structures. In recent years, attention mechanism, which is brilliant in the fields of natural language processing and computer vision, is introduced to GNNs to adaptively select the discriminative features and automatically filter the noisy information. To the best of our knowledge, due to the fast-paced advances in this domain, a systematic overview of attention-based GNNs is still missing. To fill this gap, this paper aims to provide a comprehensive survey on recent advances in attention-based GNNs. Firstly, we propose a novel two-level taxonomy for attention-based GNNs from the perspective of development history and architectural perspectives. Specifically, the upper level reveals the three developmental stages of attention-based GNNs, including graph recurrent attention networks, graph attention networks, and graph transformers. The lower level focuses on various typical architectures of each stage. Secondly, we review these attention-based methods following the proposed taxonomy in detail and summarize the advantages and disadvantages of various models. A model characteristics table is also provided for a more comprehensive comparison. Thirdly, we share our thoughts on some open issues and future directions of attention-based GNNs. We hope this survey will provide researchers with an up-to-date reference regarding applications of attention-based GNNs. In addition, to cope with the rapid development in this field, we intend to share the relevant latest papers as an open resource at https://github.com/sunxiaobei/awesome-attention-based-gnns.

Problem

• attention-based GNNs
• graph neural networks
• attention mechanism
• systematic survey
• model taxonomy

Innovation

• attention-based GNNs
• taxonomy
• graph transformers
• graph attention networks
• survey