UniTraj: Learning a Universal Trajectory Foundation Model from Billion-Scale Worldwide Traces

2024-11-06

arXiv.org

AI Summary

Existing trajectory modeling approaches suffer from task specialization, region dependency, and data sensitivity. To address these limitations, we propose UniTraj—the first universal trajectory foundation model—featuring a task-adaptive architecture and geography-robust representation learning for unified cross-task and cross-regional modeling. We introduce WorldTrace, an open-source, multi-national benchmark comprising 2.45 million trajectories and billions of trajectory points, and design a multi-scale resampling scheme alongside spatiotemporal masking for pretraining within a Transformer framework to effectively capture long-range spatiotemporal dependencies. Experiments demonstrate that UniTraj consistently outperforms state-of-the-art methods across diverse tasks—including trajectory prediction, completion, and classification. Moreover, it exhibits exceptional generalization and robustness on real-world trajectory data from 70 countries, significantly improving modeling performance in low-quality and sparse regions.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models
• Computer Vision: Motion & Tracking

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Human trajectory modeling is essential for deciphering movement patterns and supporting advanced applications across various domains. However, existing methods are often tailored to specific tasks and regions, resulting in limitations related to task specificity, regional dependency, and data quality sensitivity. Addressing these challenges requires a universal human trajectory foundation model capable of generalizing and scaling across diverse tasks and geographic contexts. To this end, we propose UniTraj, a Universal human Trajectory foundation model that is task-adaptive, region-independent, and highly generalizable. To further enhance performance, we construct WorldTrace, the first large-scale, high-quality, globally distributed dataset sourced from open web platforms, encompassing 2.45 million trajectories with billions of points across 70 countries. Through multiple resampling and masking strategies designed for pre-training, UniTraj effectively overcomes geographic and task constraints, adapting to heterogeneous data quality. Extensive experiments across multiple trajectory analysis tasks and real-world datasets demonstrate that UniTraj consistently outperforms existing approaches in terms of scalability and adaptability. These results underscore the potential of UniTraj as a versatile, robust solution for a wide range of trajectory analysis applications, with WorldTrace serving as an ideal but non-exclusive foundation for training.

Problem

• Building universal trajectory model overcoming task specificity limitations
• Addressing regional dependency through worldwide diverse trajectory dataset
• Solving data sensitivity with robust pre-training for heterogeneous quality

Innovation

• WorldTrace dataset with billion-scale GPS points across 70 countries
• Adaptive Trajectory Resampling and Self-supervised Trajectory Masking pre-training
• Flexible model architecture capturing complex movement patterns