TrajMoE: Spatially-Aware Mixture of Experts for Unified Human Mobility Modeling

2025-05-24

AI Summary

Cross-city human mobility modeling faces significant generalization challenges due to inconsistent spatial representations and heterogeneous mobility patterns across cities. Method: We propose SAMoE—the first Spatially-Aware Mixture-of-Experts Transformer foundation model—designed to jointly address spatial semantic misalignment and city-specific behavioral diversity. SAMoE encodes point-of-interest (POI) functional semantics to achieve cross-city spatial semantic alignment and integrates a city-scenario-adaptive architecture comprising shared experts (for transferable knowledge) and domain experts (for city-specific patterns). Contribution/Results: Leveraging a lightweight fine-tuning paradigm, SAMoE achieves state-of-the-art performance with only one round of fine-tuning on just 5% of target-city data, yielding a 27% relative improvement over strong baselines in cross-city transfer tasks. Notably, it outperforms fully trained models, demonstrating superior efficiency and scalability. SAMoE establishes a novel, generalizable paradigm for large-scale, extensible city-level mobility modeling.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Search and Optimization: Sampling/Simulation-based Search
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Modeling human mobility across diverse cities is essential for applications such as urban planning, transportation optimization, and personalized services. However, generalization remains challenging due to heterogeneous spatial representations and mobility patterns across cities. Existing methods typically rely on numerical coordinates or require training city-specific models, limiting their scalability and transferability. We propose TrajMoE, a unified and scalable model for cross-city human mobility modeling. TrajMoE addresses two key challenges: (1) inconsistent spatial semantics across cities, and (2) diverse urban mobility patterns. To tackle these, we begin by designing a spatial semantic encoder that learns transferable location representations from POI-based functional semantics and visit patterns. Furthermore, we design a Spatially-Aware Mixture-of-Experts (SAMoE) Transformer that injects structured priors into experts specialized in distinct mobility semantics, along with a shared expert to capture city-invariant patterns and enable adaptive cross-city generalization. Extensive experiments demonstrate that TrajMoE achieves up to 27% relative improvement over competitive mobility foundation models after only one epoch of fine-tuning, and consistently outperforms full-data baselines using merely 5% of target city data. These results establish TrajMoE as a significant step toward realizing a truly generalizable, transferable, and pretrainable foundation model for human mobility.

Problem

• Modeling human mobility across diverse cities with heterogeneous spatial representations
• Addressing inconsistent spatial semantics and diverse urban mobility patterns
• Developing a scalable, transferable model for cross-city mobility generalization

Innovation

• Spatial semantic encoder for transferable location representations
• Spatially-Aware Mixture-of-Experts Transformer for mobility semantics
• Shared expert captures city-invariant patterns