Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications

2025-10-23

AI Summary

This work addresses the complementary deficiencies of large language models (LLMs)—weak structured reasoning—and text-attributed graphs (TAGs)—shallow semantic representation—by proposing a bidirectional, synergistic LLM–TAG fusion framework. Methodologically, it introduces, for the first time, a unified taxonomy from an orchestration perspective, distinguishing “LLM for TAG” and “TAG for LLM” pathways, and systematically designs sequential, parallel, and multi-module integration strategies. The framework integrates graph neural networks, prompt engineering, parameter-efficient fine-tuning, and graph pretraining to jointly model semantic and structural information. Contributions include: (1) establishing the first theoretical framework for LLM–TAG fusion; (2) achieving significant improvements in interpretability, logical reasoning, and cross-domain generalization across recommendation systems, biomedical analysis, and knowledge-based question answering; and (3) curating a comprehensive survey of mainstream datasets and empirical benchmarks.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) have achieved remarkable success in natural language processing through strong semantic understanding and generation. However, their black-box nature limits structured and multi-hop reasoning. In contrast, Text-Attributed Graphs (TAGs) provide explicit relational structures enriched with textual context, yet often lack semantic depth. Recent research shows that combining LLMs and TAGs yields complementary benefits: enhancing TAG representation learning and improving the reasoning and interpretability of LLMs. This survey provides the first systematic review of LLM--TAG integration from an orchestration perspective. We introduce a novel taxonomy covering two fundamental directions: LLM for TAG, where LLMs enrich graph-based tasks, and TAG for LLM, where structured graphs improve LLM reasoning. We categorize orchestration strategies into sequential, parallel, and multi-module frameworks, and discuss advances in TAG-specific pretraining, prompting, and parameter-efficient fine-tuning. Beyond methodology, we summarize empirical insights, curate available datasets, and highlight diverse applications across recommendation systems, biomedical analysis, and knowledge-intensive question answering. Finally, we outline open challenges and promising research directions, aiming to guide future work at the intersection of language and graph learning.

Problem

• Integrating LLMs with text-attributed graphs for enhanced reasoning
• Addressing semantic depth limitations in graphs using language models
• Developing orchestration frameworks to combine structural and semantic learning

Innovation

• LLMs enrich graph-based tasks with semantics
• Graph structures improve LLM reasoning capabilities
• Orchestration strategies include sequential and parallel frameworks