Trip+: Benchmarking Agents in Personalized Interactive Travel Planning

2026-06-19

AI Summary

Existing benchmarks struggle to comprehensively evaluate travel-planning agents across the interplay of dynamic interaction, personalized preferences, and itinerary feasibility. To address this gap, this work proposes Trip+, a novel benchmark that unifies personalized preferences, multi-turn dynamic interaction, and minute-level itinerary planning within a single evaluative framework, while incorporating subjective experience metrics such as fatigue. The benchmark leverages large language model (LLM)-driven agents to generate and dynamically replan itineraries, and employs an LLM-based simulator for end-to-end assessment of the overall travel experience. Experiments across 18 language models reveal that, although current models can produce technically feasible itineraries, they frequently overlook user preferences, resulting in suboptimal experiences and excessive fatigue.

Abstract

Interactive travel planning has become a popular use case for language models. Agents are deployed to manage evolving preferences and unexpected disruptions over multiple turns. Such settings require models to make complex, profile-conditioned planning decisions. However, existing benchmarks often evaluate feasibility, personalization, or interaction in relatively isolated settings. We therefore introduce Trip+ to measure the ability of agents to plan travel holistically. In Trip+, given traveler profiles and dynamic interactions, agents must generate and revise minute-level itineraries. End-to-end traveler experiences are evaluated via an LLM-based simulator, enabling the assessment of subjective metrics like fatigue. Our scenarios range from simple request resolutions to complex environment-driven replanning. We evaluate 18 LMs and find a consistent gap in experiential quality. Models favor technically feasible but exhausting itineraries that diverge sharply from profiled traveler preferences.

Problem

• interactive travel planning
• personalization
• agent benchmarking
• itinerary generation
• traveler preferences

Innovation

• personalized travel planning
• interactive agent benchmarking
• LLM-based simulation
• minute-level itinerary generation
• experiential quality evaluation