TAG-DLM: Diffusion Language Models for Text-Attributed Graph Learning

2026-06-30

AI Summary

Existing approaches struggle to effectively jointly model linguistic semantics and graph structure in textual attributed graphs. This work proposes a unified framework that linearizes graph neighborhoods into token sequences and integrates a topological attention mask into a masked diffusion language model, thereby unifying bidirectional attention with generative decoding to enable end-to-end co-optimization of textual reasoning and graph message passing. Through a structure-aware prompting mechanism, the method supports node classification, link prediction, and cross-dataset transfer without task-specific fine-tuning. Evaluated on three benchmarks, it significantly outperforms baselines based on graph neural networks, graph Transformers, and large language models, achieving performance gains of up to 3.9 percentage points.

Abstract

Text-attributed graphs (TAGs), where each node carries a natural language description, require models to jointly reason over text and graph topology. Existing approaches often handle the two modalities separately: graph neural networks operate on shallow text features, while hybrids of LLMs and graphs use the language model mainly as a text encoder and delegate structure learning to a separate graph module. We propose method that unifies textual reasoning and graph message passing within a masked diffusion language model, a language model with bidirectional attention and generative decoding. For each graph instance, method linearises a sampled local neighbourhood into a token sequence and injects graph structure through a topology attention mask, which realises message passing over the graph. Because the diffusion language model can both interpret and generate text, the method adapts to different tasks simply by changing the prompt, supporting node classification, link prediction, and cross-dataset transfer with no target-specific fine-tuning. Experiments show that method outperforms graph neural networks, graph transformers, and LLM-based baselines on all three TAG benchmarks across two tasks, improving over the strongest baseline by up to 3.9 points.

Problem

• text-attributed graphs
• multimodal reasoning
• graph neural networks
• language models
• graph topology

Innovation

• Diffusion Language Model
• Text-Attributed Graph
• Topology Attention Mask
• Prompt-based Adaptation
• Graph Message Passing