Efficient Temporal Tokenization for Mobility Prediction with Large Language Models

2025-07-18

AI Summary

To address the high computational cost, inefficiency in modeling long sequences, and limited accuracy of large language models (LLMs) in human mobility prediction, this paper proposes RHYTHM—a lightweight, hierarchy-aware framework. Methodologically, RHYTHM introduces (1) hierarchical temporal tokenization, segmenting trajectories into daily chunks and explicitly capturing intra-day, inter-day, and weekly temporal dependencies; and (2) LLM backbone freezing with precomputed, reusable prompt embeddings to drastically reduce training overhead. By integrating discrete trajectory encoding with hierarchical attention mechanisms, RHYTHM achieves consistent improvements across three real-world datasets: +2.4% overall accuracy, +5.0% accuracy specifically on weekend predictions, and −24.6% training time—demonstrating a favorable trade-off between efficiency and predictive performance.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We introduce RHYTHM (Reasoning with Hierarchical Temporal Tokenization for Human Mobility), a framework that leverages large language models (LLMs) as spatio-temporal predictors and trajectory reasoners. RHYTHM partitions trajectories into daily segments encoded as discrete tokens with hierarchical attention, capturing both daily and weekly dependencies while substantially reducing the sequence length. Token representations are enriched with pre-computed prompt embeddings via a frozen LLM, enhancing the model's ability to capture interdependencies without extensive computational overhead. By freezing the LLM backbone, RHYTHM achieves significant computational efficiency. Evaluation on three real-world datasets demonstrates a 2.4% improvement in accuracy, 5.0% increase on weekends, and 24.6% reduction in training time compared to state-of-the-art methods.

Problem

• Efficient temporal tokenization for mobility prediction using LLMs
• Reducing sequence length with hierarchical attention encoding
• Enhancing accuracy and efficiency in mobility trajectory prediction

Innovation

• Hierarchical temporal tokenization reduces sequence length
• Pre-computed prompt embeddings enhance interdependency capture
• Frozen LLM backbone ensures computational efficiency