Search papers, labs, and topics across Lattice.
This paper introduces ProfiLLM, a novel data pipeline that leverages Large Language Models (LLMs) for utility-aligned user profiling in industrial ride-hailing dispatch systems. By addressing the challenges of scaling user profiling with limited interaction data and extensive context windows, ProfiLLM employs a two-module approach: Tool-Augmented Global Knowledge Mining and Utility-Aligned Profile Exploration, which collectively enhance prediction accuracy and operational efficiency. The deployment of ProfiLLM on DiDi's production dispatcher resulted in significant improvements, including a 6.14% increase in AUC and a 4.35% gain in gross merchandise value (GMV) during simulations.
ProfiLLM achieves a remarkable 6.14% boost in prediction accuracy for ride-hailing dispatch by transforming user behavior into actionable profiles using LLMs.
Bringing Large Language Models (LLMs) into industrial ride-hailing dispatch as semantic feature extractors over platform-scale behavioral logs is a compelling but under-explored data systems problem. Production matching pipelines remain dominated by structured numerical features, yet decisive behavioral signals (e.g., a driver's habitual aversion to certain regions) are inherently contextual and naturally expressible as LLM-generated user profiles. However, scaling such profiling to a live, millisecond-latency dispatcher faces three intertwined constraints rarely addressed together: on a platform with millions of daily orders, logs exceed any LLM's context window by orders of magnitude; most users are long-tail, with too few interactions for per-user profiling; and surface-fluent profiles do not necessarily improve downstream prediction utility. We present ProfiLLM, an agentic LLM data pipeline that operationalizes utility-aligned user profiling for production matching systems through two modules. (1) Tool-Augmented Global Knowledge Mining equips an LLM agent with 27 analytical tools to mine platform-scale data, producing reusable global knowledge, adaptive user clustering rules, and region-level supply-demand priors. (2) Utility-Aligned Profile Exploration generates multiple candidate profiles per cluster, evaluates them via a lightweight downstream utility proxy, iteratively refines the best candidates and constructs preference pairs for DPO fine-tuning. Deployed on DiDi's production dispatcher, ProfiLLM achieves up to +6.14% relative AUC improvement in outcome prediction, up to +4.35% GMV gain in dispatching simulation, and consistent improvements in a 14-day online A/B test including +0.47% GMV, +0.33% Completion Rate, and -0.82% Cancel-Before-Accept rate.
