BERT4Traj: Transformer Based Trajectory Reconstruction for Sparse Mobility Data

2025-07-03

AI Summary

Human mobility trajectory data are often severely incomplete due to sparse GPS sampling or coarse-grained cellular call detail records (CDRs), limiting their utility in public health and urban planning. To address this, we propose the first trajectory reconstruction framework grounded in BERT-style masked language modeling and self-attention mechanisms. Our method innovatively fuses spatiotemporal embeddings with heterogeneous contextual features—including demographic attributes and geographic anchors—enabling end-to-end learning of latent mobility sequences via masked token prediction. Built upon the Transformer architecture, the model is rigorously evaluated on real-world CDR and GPS traces from Kampala, Uganda. It significantly outperforms established baselines—including Markov chains, k-nearest neighbors (k-NN), RNNs, and LSTMs—in reconstructing continuous, fine-grained, and semantically plausible trajectories. The framework establishes a novel paradigm for modeling sparse mobility data, offering high-fidelity trajectory recovery with broad implications for downstream applications in epidemiology and smart city analytics.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Feature Construction/Reformulation
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Understanding human mobility is essential for applications in public health, transportation, and urban planning. However, mobility data often suffers from sparsity due to limitations in data collection methods, such as infrequent GPS sampling or call detail record (CDR) data that only capture locations during communication events. To address this challenge, we propose BERT4Traj, a transformer based model that reconstructs complete mobility trajectories by predicting hidden visits in sparse movement sequences. Inspired by BERT's masked language modeling objective and self_attention mechanisms, BERT4Traj leverages spatial embeddings, temporal embeddings, and contextual background features such as demographics and anchor points. We evaluate BERT4Traj on real world CDR and GPS datasets collected in Kampala, Uganda, demonstrating that our approach significantly outperforms traditional models such as Markov Chains, KNN, RNNs, and LSTMs. Our results show that BERT4Traj effectively reconstructs detailed and continuous mobility trajectories, enhancing insights into human movement patterns.

Problem

• Reconstructs complete mobility trajectories from sparse data
• Predicts hidden visits in sparse movement sequences
• Enhances human movement pattern insights

Innovation

• Transformer model for sparse trajectory reconstruction
• Uses spatial and temporal embeddings with background features
• Outperforms traditional Markov Chains and RNNs