TP-RAG: Benchmarking Retrieval-Augmented Large Language Model Agents for Spatiotemporal-Aware Travel Planning

2025-04-11

AI Summary

Existing large language models for travel planning lack effective evaluation of spatiotemporal plausibility, route efficiency, and dynamic adaptability. To address this, we propose TP-RAG—the first spatiotemporally aware, retrieval-augmented benchmark for travel planning—comprising 2,348 real-world queries, 85,000 fine-grained points of interest (POIs), and 18,000 high-quality reference trajectories. Methodologically, we introduce a spatiotemporal rationality-driven evaluation paradigm and EvoRAG, an evolutionary retrieval-augmented framework integrating multi-source trajectory references, spatiotemporal constraint modeling, fine-grained POI semantic annotation, and evolutionary prompt optimization. Experimental results demonstrate that EvoRAG significantly improves spatial route efficiency and POI selection rationality, achieves superior spatiotemporal compliance over baselines, and reduces commonsense errors—establishing it as the current state-of-the-art method for travel planning agents.

Technology Category

• Planning, Routing, and Scheduling: Temporal Planning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models (LLMs) have shown promise in automating travel planning, yet they often fall short in addressing nuanced spatiotemporal rationality. While existing benchmarks focus on basic plan validity, they neglect critical aspects such as route efficiency, POI appeal, and real-time adaptability. This paper introduces TP-RAG, the first benchmark tailored for retrieval-augmented, spatiotemporal-aware travel planning. Our dataset includes 2,348 real-world travel queries, 85,575 fine-grain annotated POIs, and 18,784 high-quality travel trajectory references sourced from online tourist documents, enabling dynamic and context-aware planning. Through extensive experiments, we reveal that integrating reference trajectories significantly improves spatial efficiency and POI rationality of the travel plan, while challenges persist in universality and robustness due to conflicting references and noisy data. To address these issues, we propose EvoRAG, an evolutionary framework that potently synergizes diverse retrieved trajectories with LLMs' intrinsic reasoning. EvoRAG achieves state-of-the-art performance, improving spatiotemporal compliance and reducing commonsense violation compared to ground-up and retrieval-augmented baselines. Our work underscores the potential of hybridizing Web knowledge with LLM-driven optimization, paving the way for more reliable and adaptive travel planning agents.

Problem

• Addressing spatiotemporal rationality in travel planning with LLMs
• Evaluating route efficiency, POI appeal, and real-time adaptability
• Improving universality and robustness in dynamic travel planning

Innovation

• TP-RAG benchmark for spatiotemporal-aware travel planning
• EvoRAG framework synergizes trajectories with LLM reasoning
• Dynamic planning using annotated POIs and trajectory references