Building a Foundation Model for Trajectory from Scratch

2025-11-25

AI Summary

A systematic methodology and reproducible practice for building mobile trajectory foundation models (TrajFMs) from scratch remains lacking. Method: We propose a lightweight TrajFM construction paradigm based on the GPT-2 architecture, introducing time-series patching to trajectory modeling for the first time, and designing spatiotemporal-aware data encoding and positional embedding strategies. We release a fully modular, open-source implementation. Within a unified evaluation framework, we comparatively analyze TrajFM, TrajGPT, and other state-of-the-art methods, clarifying fundamental differences in modeling assumptions, input representations, and training objectives. Contribution/Results: This work bridges critical pedagogical and engineering gaps in trajectory AI, substantially enhancing transparency, auditability, and reproducibility in TrajFM development. It provides a standardized technical reference for the SIGSPATIAL and broader spatial AI communities.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Computer Vision: Motion & Tracking

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Foundation models are transformative in artificial intelligence, but building them from scratch, especially for mobility trajectories, is not yet clear or documented. This tutorial bridges this gap by demonstrating the steps and code of a minimal implementation of a trajectory-focused foundation model starting from GPT-2. Through a concise, step-by-step, code-driven process, we demonstrate adapting GPT-2 for spatiotemporal data. We then review and compare representative trajectory foundation models, such as TrajFM and TrajGPT, highlighting their architectural innovations and differences. Additionally, we introduce complementary techniques from related domains, like TimesFM's patching approach. Targeted at researchers and practitioners, this tutorial aims to explain the concepts and terminology of foundation models, at the implementation level. We find it timely and indispensable to create this educational material in order to support the SIGSPATIAL community in building and evaluating mobility foundation models, enhancing both research clarity and peer-review effectiveness in mobility AI.

Problem

• Building trajectory foundation models from scratch lacks clarity
• Adapting GPT-2 architecture for spatiotemporal trajectory data processing
• Comparing architectural innovations across different trajectory foundation models

Innovation

• Adapting GPT-2 for spatiotemporal trajectory data
• Comparing TrajFM and TrajGPT architectural innovations
• Introducing TimesFM patching approach for trajectories