RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility

2025-09-27

AI Summary

Human mobility trajectory prediction faces challenges in modeling long-range temporal dependencies and capturing multi-scale periodicities (e.g., daily and weekly cycles). To address these, we propose the first unified spatiotemporal reasoning framework based on large language models (LLMs). Our method introduces a hierarchical temporal tokenization scheme that discretizes trajectories into daily tokens; incorporates dual-granularity attention mechanisms—operating at both daily and weekly levels—to explicitly model periodic dependencies; and freezes the LLM backbone while precomputing prompt embeddings to drastically reduce computational overhead. Evaluated on three real-world datasets, our approach achieves new state-of-the-art performance: +2.4% overall accuracy, +5.0% accuracy on weekend predictions, and −24.6% training time. The core contribution lies in successfully adapting LLMs to mobility trajectory modeling, enabling efficient and interpretable multi-scale spatiotemporal reasoning for the first time.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Predicting human mobility is inherently challenging due to complex long-range dependencies and multi-scale periodic behaviors. To address this, we introduce RHYTHM (Reasoning with Hierarchical Temporal Tokenization for Human Mobility), a unified framework that leverages large language models (LLMs) as general-purpose spatio-temporal predictors and trajectory reasoners. Methodologically, RHYTHM employs temporal tokenization to partition each trajectory into daily segments and encode them as discrete tokens with hierarchical attention that captures both daily and weekly dependencies, thereby significantly reducing the sequence length while preserving cyclical information. Additionally, we enrich token representations by adding pre-computed prompt embeddings for trajectory segments and prediction targets via a frozen LLM, and feeding these combined embeddings back into the LLM backbone to capture complex interdependencies. Computationally, RHYTHM freezes the pretrained LLM's backbone to reduce attention complexity and memory cost. We evaluate our model against state-of-the-art methods using three real-world datasets. Notably, RHYTHM achieves a 2.4% improvement in overall accuracy, a 5.0% increase on weekends, and a 24.6% reduction in training time. Code is publicly available at https://github.com/he-h/rhythm.

Problem

• Predicting human mobility with complex long-range dependencies and multi-scale periodic behaviors
• Reducing sequence length while preserving cyclical trajectory information through tokenization
• Improving prediction accuracy and reducing training time for mobility forecasting

Innovation

• Hierarchical temporal tokenization reduces sequence length
• Frozen LLM backbone with prompt embeddings captures dependencies
• Combined embeddings enhance trajectory reasoning accuracy