🤖 AI Summary
In task-oriented dialogue, large language models (LLMs) frequently deviate from prescribed workflows when handling long-horizon, conditional tasks that depend on external tools and user-specific context. This paper proposes WARPP, a training-free modular framework leveraging multi-agent collaboration and runtime personalization to enhance task adherence. Its core innovation is a parallel Personalizer agent that dynamically prunes conditional branches and customizes execution paths in real time, substantially reducing inference overhead and improving tool-call accuracy. WARPP integrates conditional dependency analysis, synthetic data simulation, and LLM-driven automated evaluation. Experiments across five user intents in banking, flight booking, and healthcare domains demonstrate that as task complexity increases, parameter fidelity and tool accuracy improve significantly, while average token consumption decreases.
📝 Abstract
Large language models (LLMs) are increasingly applied in task-oriented dialogue (TOD) systems but often struggle with long, conditional workflows that involve external tool calls and depend on user-specific information. We present Workflow Adherence via Runtime Parallel Personalization, or WARPP, a training-free, modular framework that combines multi-agent orchestration with runtime personalization to improve workflow adherence in LLM-based systems. By dynamically pruning conditional branches based on user attributes, the framework reduces reasoning overhead and narrows tool selection at runtime. WARPP deploys a parallelized architecture where a dedicated Personalizer agent operates alongside modular, domain-specific agents to dynamically tailor execution paths in real time. The framework is evaluated across five representative user intents of varying complexity within three domains: banking, flights, and healthcare. Our evaluation leverages synthetic datasets and LLM-powered simulated users to test scenarios with conditional dependencies. Our results demonstrate that WARPP outperforms both the non-personalized method and the ReAct baseline, achieving increasingly larger gains in parameter fidelity and tool accuracy as intent complexity grows, while also reducing average token usage, without any additional training.