ProfiLLM: Utility-Aligned Agentic User Profiling for Industrial Ride-Hailing Dispatch

2026-06-17

AI Summary

This work addresses the challenges of millisecond-level latency, long-tailed sparse interactions, and ultra-large-scale data in user profiling for industrial ride-hailing dispatch systems. We propose the first scalable large language model (LLM)–based user profiling paradigm that integrates utility alignment with an agent architecture. Our approach employs a two-stage pipeline—tool-augmented global knowledge mining followed by utility-aligned profile exploration—incorporating adaptive clustering, regional supply-demand priors, multi-candidate generation, and lightweight utility proxy evaluation, with preference alignment fine-tuned via Direct Preference Optimization (DPO). Evaluated on DiDi’s production system, the method significantly enhances downstream performance: prediction AUC improves by up to 6.14%, simulated GMV increases by 4.35%, and online A/B tests demonstrate a 0.47% GMV gain, a 0.33% rise in order completion rate, and a 0.82% reduction in pre-acceptance cancellation rate.

Abstract

Bringing Large Language Models (LLMs) into industrial ride-hailing dispatch as semantic feature extractors over platform-scale behavioral logs is a compelling but under-explored data systems problem. Production matching pipelines remain dominated by structured numerical features, yet decisive behavioral signals (e.g., a driver's habitual aversion to certain regions) are inherently contextual and naturally expressible as LLM-generated user profiles. However, scaling such profiling to a live, millisecond-latency dispatcher faces three intertwined constraints rarely addressed together: on a platform with millions of daily orders, logs exceed any LLM's context window by orders of magnitude; most users are long-tail, with too few interactions for per-user profiling; and surface-fluent profiles do not necessarily improve downstream prediction utility. We present ProfiLLM, an agentic LLM data pipeline that operationalizes utility-aligned user profiling for production matching systems through two modules. (1) Tool-Augmented Global Knowledge Mining equips an LLM agent with 27 analytical tools to mine platform-scale data, producing reusable global knowledge, adaptive user clustering rules, and region-level supply-demand priors. (2) Utility-Aligned Profile Exploration generates multiple candidate profiles per cluster, evaluates them via a lightweight downstream utility proxy, iteratively refines the best candidates and constructs preference pairs for DPO fine-tuning. Deployed on DiDi's production dispatcher, ProfiLLM achieves up to +6.14% relative AUC improvement in outcome prediction, up to +4.35% GMV gain in dispatching simulation, and consistent improvements in a 14-day online A/B test including +0.47% GMV, +0.33% Completion Rate, and -0.82% Cancel-Before-Accept rate.

Problem

• user profiling
• large language models
• ride-hailing dispatch
• utility alignment
• industrial-scale systems

Innovation

• Utility-Aligned Profiling
• Agentic LLM
• Tool-Augmented Knowledge Mining
• DPO Fine-tuning
• Industrial Ride-Hailing Dispatch