Are LLM-Enhanced Graph Neural Networks Robust against Poisoning Attacks?

2026-03-27

AI Summary

This work addresses the lack of systematic evaluation of the robustness of large language model (LLM)-enhanced graph neural networks (GNNs) under joint structural and textual poisoning attacks. We present the first multidimensional benchmarking framework tailored for the post-LLM era, encompassing four real-world datasets, eight LLM-based feature enhancers, three GNN backbones, six structural attack variants (targeted and untargeted), and three text-level poisoning strategies operating at character, word, and sentence levels. Additionally, we introduce a novel composite attack methodology and a graph purification-based defense mechanism. Experimental results demonstrate that LLM-GNNs substantially outperform shallow embedding baselines, achieving higher accuracy and lower relative degradation accuracy (RDA), thereby revealing that effectively encoding both structural and label information is pivotal to enhancing model robustness.

Abstract

Large Language Models (LLMs) have advanced Graph Neural Networks (GNNs) by enriching node representations with semantic features, giving rise to LLM-enhanced GNNs that achieve notable performance gains. However, the robustness of these models against poisoning attacks, which manipulate both graph structures and textual attributes during training, remains unexplored. To bridge this gap, we propose a robustness assessment framework that systematically evaluates LLM-enhanced GNNs under poisoning attacks. Our framework enables comprehensive evaluation across multiple dimensions. Specifically, we assess 24 victim models by combining eight LLM- or Language Model (LM)-based feature enhancers with three representative GNN backbones. To ensure diversity in attack coverage, we incorporate six structural poisoning attacks (both targeted and non-targeted) and three textual poisoning attacks operating at the character, word, and sentence levels. Furthermore, we employ four real-world datasets, including one released after the emergence of LLMs, to avoid potential ground truth leakage during LLM pretraining, thereby ensuring fair evaluation. Extensive experiments show that LLM-enhanced GNNs exhibit significantly higher accuracy and lower Relative Drop in Accuracy (RDA) than a shallow embedding-based baseline across various attack settings. Our in-depth analysis identifies key factors that contribute to this robustness, such as the effective encoding of structural and label information in node representations. Based on these insights, we outline future research directions from both offensive and defensive perspectives, and propose a new combined attack along with a graph purification defense. To support future research, we release the source code of our framework at~\url{https://github.com/CyberAlSec/LLMEGNNRP}.

Problem

• LLM-enhanced GNNs
• poisoning attacks
• robustness
• graph neural networks
• adversarial attacks

Innovation

• LLM-enhanced GNNs
• poisoning attacks
• robustness evaluation
• graph-text joint poisoning
• graph purification defense