Strada-LLM: Graph LLM for traffic prediction

2024-10-28

arXiv.org

AI Summary

Traffic forecasting faces challenges including significant distributional shifts across heterogeneous multi-source data and poor generalization under few-shot settings; existing large language model (LLM) approaches lack joint modeling of graph structure and spatiotemporal dynamics. To address this, we propose Graph-LLM—the first graph-aware LLM framework—featuring: (1) a novel graph-structure-enhanced LLM architecture that injects adjacency node states as spatial covariates; (2) a lightweight few-shot domain adaptation mechanism enabling rapid adaptation to unseen regions with only 3–5 samples; and (3) plug-and-play compatibility with diverse LLM backbones. By integrating GNN-based feature encoding, spatiotemporal covariate fusion, prompt-based fine-tuning, and probabilistic decoding, Graph-LLM achieves substantial improvements over state-of-the-art LLMs and GNNs across multiple real-world datasets. In few-shot scenarios, it reduces MAE by 12.7%, demonstrating both high accuracy and interpretability.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Planning, Routing, and Scheduling: Planning with Language Models
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Traffic prediction is a vital component of intelligent transportation systems. By reasoning about traffic patterns in both the spatial and temporal dimensions, accurate and interpretable predictions can be provided. A considerable challenge in traffic prediction lies in handling the diverse data distributions caused by vastly different traffic conditions occurring at different locations. LLMs have been a dominant solution due to their remarkable capacity to adapt to new datasets with very few labeled data samples, i.e., few-shot adaptability. However, existing forecasting techniques mainly focus on extracting local graph information and forming a text-like prompt, leaving LLM- based traffic prediction an open problem. This work presents a probabilistic LLM for traffic forecasting with three highlights. We propose a graph-aware LLM for traffic prediction that considers proximal traffic information. Specifically, by considering the traffic of neighboring nodes as covariates, our model outperforms the corresponding time-series LLM. Furthermore, we adopt a lightweight approach for efficient domain adaptation when facing new data distributions in few-shot fashion. The comparative experiment demonstrates the proposed method outperforms the state-of-the-art LLM-based methods and the traditional GNN- based supervised approaches. Furthermore, Strada-LLM can be easily adapted to different LLM backbones without a noticeable performance drop.

Problem

• Addresses traffic prediction in intelligent transportation systems
• Handles diverse data distributions across different locations
• Improves few-shot adaptability in traffic forecasting models

Innovation

• Graph-aware LLM for traffic prediction
• Lightweight domain adaptation approach
• Proximal traffic information consideration