Unify Graph Learning with Text: Unleashing LLM Potentials for Session Search

2024-05-13

The Web Conference

AI Summary

Existing conversational search methods inadequately model graph-structured interactions and fail to fuse fine-grained textual semantics with structural information. Method: We propose the Symbolic Graph Ranker (SGR), which (i) formalizes dialogue history as a conversational graph and symbolically encodes it—via an interpretable, grammar-based syntax—into LLM-processable textual sequences; (ii) introduces multi-granularity self-supervised objectives (link prediction, node generation, contrastive generation) to explicitly guide LLMs in learning graph topology; and (iii) bypasses reliance on generic document embeddings to jointly model lexical semantics and graph structure. Contribution/Results: SGR achieves significant improvements over state-of-the-art methods on the AOL and Tiangong-ST benchmarks, demonstrating both the effectiveness and generalizability of graph-text joint modeling for conversational search.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Session search involves a series of interactive queries and actions to fulfill user's complex information need. Current strategies typically prioritize sequential modeling for deep semantic understanding, overlooking the graph structure in interactions. While some approaches focus on capturing structural information, they use a generalized representation for documents, neglecting the word-level semantic modeling. In this paper, we propose Symbolic Graph Ranker (SGR), which aims to take advantage of both text-based and graph-based approaches by leveraging the power of recent Large Language Models (LLMs). Concretely, we first introduce a set of symbolic grammar rules to convert session graph into text. This allows integrating session history, interaction process, and task instruction seamlessly as inputs for the LLM. Moreover, given the natural discrepancy between LLMs pre-trained on textual corpora, and the symbolic language we produce using our graph-to-text grammar, our objective is to enhance LLMs' ability to capture graph structures within a textual format. To achieve this, we introduce a set of self-supervised symbolic learning tasks including link prediction, node content generation, and generative contrastive learning, to enable LLMs to capture the topological information from coarse-grained to fine-grained. Experiment results and comprehensive analysis on two benchmark datasets, AOL and Tiangong-ST, confirm the superiority of our approach. Our paradigm also offers a novel and effective methodology that bridges the gap between traditional search strategies and modern LLMs.

Problem

• Bridging graph structure and text modeling in session search
• Enhancing LLMs to capture graph structures via symbolic learning
• Integrating session history and interaction processes for LLM inputs

Innovation

• Symbolic grammar rules convert session graph to text
• Self-supervised tasks enhance LLM graph structure capture
• Integrates session history and interaction process seamlessly