Dynamic Text Bundling Supervision for Zero-Shot Inference on Text-Attributed Graphs

2025-05-23

AI Summary

To address the semantic insufficiency and unreliable predictions of large language models (LLMs) in zero-shot inference on text-attributed graphs (TAGs)—stemming from their disconnection from graph topology—this paper proposes a dynamic text bundling supervision paradigm. Our method generates bundle-level labels via topology-aware text bundling, integrates proximity-driven dynamic text sampling, employs GNN-LLM collaborative prompt modeling, and incorporates theoretically grounded label reliability assessment to enable adaptive noise filtering and joint optimization. Evaluated on ten benchmark datasets, our approach achieves an average 9.2% improvement in zero-shot node classification accuracy. It is the first framework to jointly enforce graph structural constraints and semantic generation capability under unified supervision, thereby significantly enhancing the generalizability and trustworthiness of LLMs in structured textual scenarios.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models (LLMs) have been used in many zero-shot learning problems, with their strong generalization ability. Recently, adopting LLMs in text-attributed graphs (TAGs) has drawn increasing attention. However, the adoption of LLMs faces two major challenges: limited information on graph structure and unreliable responses. LLMs struggle with text attributes isolated from the graph topology. Worse still, they yield unreliable predictions due to both information insufficiency and the inherent weakness of LLMs (e.g., hallucination). Towards this end, this paper proposes a novel method named Dynamic Text Bundling Supervision (DENSE) that queries LLMs with bundles of texts to obtain bundle-level labels and uses these labels to supervise graph neural networks. Specifically, we sample a set of bundles, each containing a set of nodes with corresponding texts of close proximity. We then query LLMs with the bundled texts to obtain the label of each bundle. Subsequently, the bundle labels are used to supervise the optimization of graph neural networks, and the bundles are further refined to exclude noisy items. To justify our design, we also provide theoretical analysis of the proposed method. Extensive experiments across ten datasets validate the effectiveness of the proposed method.

Problem

• LLMs struggle with isolated text attributes in graphs
• LLMs produce unreliable predictions due to information gaps
• Dynamic bundling improves supervision for graph neural networks

Innovation

• Dynamic Text Bundling Supervision (DENSE) method
• Queries LLMs with bundled texts for labels
• Supervises GNNs with refined bundle labels