LoReC: Rethinking Large Language Models for Graph Data Analysis

2026-04-20

AI Summary

Large language models (LLMs) often struggle to effectively leverage structural information in graph-based tasks, frequently underperforming compared to traditional graph neural networks (GNNs). To address this limitation, this work proposes LoReC, a plug-and-play framework that enhances LLMs’ comprehension of graph data through a three-stage “observe–remember–contrast” mechanism. Specifically, LoReC dynamically reallocates attention focus within attention layers, reinjects graph structural cues into the feed-forward network, and applies contrastive correction during decoding. Extensive experiments demonstrate that LoReC significantly outperforms existing GraphLLM approaches across multiple benchmark datasets and even surpasses canonical GNN architectures, highlighting its effectiveness in bridging the gap between language modeling and graph representation learning.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

The advent of Large Language Models (LLMs) has fundamentally reshaped the way we interact with graphs, giving rise to a new paradigm called GraphLLM. As revealed in recent studies, graph learning can benefit from LLMs. However, we observe limited benefits when we directly utilize LLMs to make predictions for graph-related tasks within GraphLLM paradigm, which even yields suboptimal results compared to conventional GNN-based approaches. Through in-depth analysis, we find this failure can be attributed to LLMs' limited capability for processing graph data and their tendency to overlook graph information. To address this issue, we propose LoReC (Look, Remember, and Contrast), a novel plug-and-play method for GraphLLM paradigm, which enhances LLM's understanding of graph data through three stages: (1) Look: redistributing attention to graph; (2) Remember: re-injecting graph information into the Feed-Forward Network (FFN); (3) Contrast: rectifying the vanilla logits produced in the decoding process. Extensive experiments demonstrate that LoReC brings notable improvements over current GraphLLM methods and outperforms GNN-based approaches across diverse datasets. The implementation is available at https://github.com/Git-King-Zhan/LoReC.

Problem

• Large Language Models
• Graph Data Analysis
• GraphLLM
• Graph Neural Networks
• Graph Information Processing

Innovation

• GraphLLM
• LoReC
• Large Language Models
• Graph Representation Learning
• Plug-and-play Method