Search papers, labs, and topics across Lattice.
This paper introduces UTP-Bench, a novel benchmark designed for uncertainty-aware travel planning that addresses the limitations of existing deterministic benchmarks by incorporating real-world travel data and stochastic conditions. By integrating empirical delay distributions and crowd-density patterns from 504 cities in India, the benchmark evaluates the robustness of travel itineraries against common disruptions. Experiments with advanced LLMs reveal significant performance gaps in generating itineraries that effectively account for transit delays and crowd variability, highlighting the need for improved models in this domain.
LLMs struggle to generate robust travel itineraries, with substantial gaps in handling delays and crowd dynamics revealed by UTP-Bench.
Large Language Models (LLMs) have recently demonstrated strong capabilities in automated travel itinerary generation. However, real- world travel planning is inherently uncertain: transportation delays, crowd fluctuations, and unexpected stochastic delays frequently inval- idate otherwise feasible schedules. Existing benchmarks like TravelPlanner and TripCraft assume deterministic environments, evaluating only static constraint satisfaction and ignoring whether generated plans remain robust when such uncertainties arise. To address this limitation, we introduce UTP-Bench1 , a large-scale benchmark for uncertainty-aware travel planning. The dataset integrates real-world travel data spanning 504 cities of India, including attractions, restau- rants, accommodations, and multi-modal trans- portation networks. To model realistic disrup- tions, UTP-Bench incorporates empirical delay distributions and crowd-density patterns col- lected from major cities, enabling evaluation of travel plans under stochastic conditions. We further propose three evaluation metrics, namely Buffer Adequacy Score (BAS), Crowd- Aware Timing Score (CATS), and Transport Delay Absorption Score (TDAS), which quan- tify the ability of generated itineraries to main- tain robustness against transit delays and crowd variability. Experiments with state-of-the-art LLMs like GPT-5, Qwen3, Mistral and Phi-4 re- veal substantial gaps between model-generated and human-authored plans, particularly in tem- poral buffering, delay-aware transportation scheduling, and crowd-sensitive planning.
