Graph2text or Graph2token: A Perspective of Large Language Models for Graph Learning

2025-01-02

AI Summary

This paper addresses the fundamental challenge that large language models (LLMs) cannot natively process graph-structured data. To bridge this gap, we propose LLM4graph—a unified framework introducing the first systematic taxonomy of Graph2Text and Graph2Token paradigms, and identifying four core challenges in graph-to-text/token conversion. Methodologically, LLM4graph integrates graph encoding, structured serialization, prompt engineering, and LLM adaptation techniques to achieve multi-granular, semantics-preserving graph representation transformation. Key contributions include: (1) a principled LLM selection guideline tailored to problem characteristics, hardware constraints, and task requirements; (2) a comprehensive survey of over 100 works in graph–LLM integration; and (3) distillation of five critical future research directions. This work establishes a theoretical foundation, technical methodology, and practical paradigm for the emerging intersection of graph learning and large language modeling.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Graphs are data structures used to represent irregular networks and are prevalent in numerous real-world applications. Previous methods directly model graph structures and achieve significant success. However, these methods encounter bottlenecks due to the inherent irregularity of graphs. An innovative solution is converting graphs into textual representations, thereby harnessing the powerful capabilities of Large Language Models (LLMs) to process and comprehend graphs. In this paper, we present a comprehensive review of methodologies for applying LLMs to graphs, termed LLM4graph. The core of LLM4graph lies in transforming graphs into texts for LLMs to understand and analyze. Thus, we propose a novel taxonomy of LLM4graph methods in the view of the transformation. Specifically, existing methods can be divided into two paradigms: Graph2text and Graph2token, which transform graphs into texts or tokens as the input of LLMs, respectively. We point out four challenges during the transformation to systematically present existing methods in a problem-oriented perspective. For practical concerns, we provide a guideline for researchers on selecting appropriate models and LLMs for different graphs and hardware constraints. We also identify five future research directions for LLM4graph.

Problem

• Large Language Models
• Complex Graph Data
• Graph Learning

Innovation

• LLM4graph
• Graph-to-Text Conversion
• Large Language Model Application