Classical Feature Embeddings Help in BERT-Based Human Mobility Prediction

2025-10-23

AI Summary

Existing mobile trajectory prediction models typically model location sequences in isolation or incorporate temporal information only superficially, neglecting the rich semantic context embedded in points of interest (POIs). To address this limitation, we propose STaBERT—a novel spatiotemporal modeling framework that jointly embeds POI semantics and fine-grained temporal features (e.g., derived time descriptors) into each location node, thereby constructing a unified, semantically enhanced spatiotemporal representation. Leveraging the BERT architecture, STaBERT enables deep sequential modeling of mobility patterns. This approach breaks from conventional decoupled spatiotemporal modeling paradigms. Empirical evaluation shows that STaBERT achieves a GEO-BLEU score of 0.75 (+41% over SOTA) on single-city prediction and 0.56 (+22%) on cross-city prediction—substantially outperforming state-of-the-art methods. These results empirically validate the critical contribution of joint semantic-temporal embedding to human mobility modeling.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Human mobility forecasting is crucial for disaster relief, city planning, and public health. However, existing models either only model location sequences or include time information merely as auxiliary input, thereby failing to leverage the rich semantic context provided by points of interest (POIs). To address this, we enrich a BERT-based mobility model with derived temporal descriptors and POI embeddings to better capture the semantics underlying human movement. We propose STaBERT (Semantic-Temporal aware BERT), which integrates both POI and temporal information at each location to construct a unified, semantically enriched representation of mobility. Experimental results show that STaBERT significantly improves prediction accuracy: for single-city prediction, the GEO-BLEU score improved from 0.34 to 0.75; for multi-city prediction, from 0.34 to 0.56.

Problem

• BERT-based mobility prediction lacks semantic POI context
• Existing models fail to integrate temporal and POI information effectively
• Current approaches underutilize rich semantic features in movement patterns

Innovation

• Integrates POI embeddings with BERT for mobility prediction
• Adds temporal descriptors to capture movement semantics
• Constructs unified semantic-temporal representations for accuracy