GeoTransformer: Enhancing Urban Forecasting with Dependency Retrieval and Geospatial Attention

2024-08-16

AI Summary

Existing urban forecasting methods fall into two categories: graph-based approaches, which rely on predefined spatial structures and suffer from poor adaptability, and region-based approaches, which struggle to incorporate global urban context and lack a unified framework to integrate the strengths of both paradigms. To address these limitations, this paper proposes the first general-purpose framework that jointly leverages high-dimensional regional embeddings and dynamic spatial modeling. Specifically, we design a retrieval-based module for adaptive selection of relevant regions and introduce a geo-spatial attention mechanism that jointly models structured topological relationships and unstructured global contextual information. Extensive experiments on GDP and ride-hailing demand forecasting demonstrate that our model significantly outperforms graph neural networks and satellite-image-based baselines, validating its superior generalizability and predictive accuracy. The framework establishes a scalable, task-adaptive paradigm for spatiotemporal urban forecasting.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent advances in urban forecasting have leveraged high-dimensional spatial data through two primary approaches: graph-based methods that rely on predefined spatial structures and region-based methods that use satellite imagery for local features. Although these methods have laid an important foundation, they struggle to integrate holistic urban information and dynamically model spatial dependencies. To address this gap, we propose GeoTransformer, a framework combining high-dimensional regional embeddings with dynamic spatial modeling. GeoTransformer features two innovations: (1) a dependency retrieval module identifying spatial dependencies to select relevant regions, and (2) a geospatial attention mechanism leveraging global urban information. These components unify structural and global urban information for better predictions. Extensive experiments on GDP and ride-share demand forecasting show that GeoTransformer outperforms baselines, highlighting its effectiveness in advancing urban forecasting tasks.

Problem

• Unified framework for high-dimensional urban forecasting
• Overcoming limitations of graph-based and region-based methods
• Integrating spatial data and task-specific dependencies

Innovation

• LLM-driven dependency retrieval for urban forecasting
• GeoTransformer with geospatial attention mechanism
• Unified framework integrating graph and region methods