AdsWorldEngine: A Self-Evolving Conversational Advertising Agent through Orchestrator and Tool Coevolution

2026-08-13

AI Summary

This study addresses the challenges of intent inference and user experience disruption in conversational advertising by proposing a self-evolving agent framework. We introduce a novel orchestrator-tool iterative co-evolution mechanism, integrated with opportunity gating and label-anchored modeling, to enable system self-optimization through supervised fine-tuning, preference construction, and a cost-sensitive GRPO algorithm. Experimental results demonstrate that this approach significantly enhances both intent recognition accuracy and ad delivery decision quality. Specifically, offline relevance and diversity improve by 80% and 60%, respectively, while online RPM increases by 22% and coverage expands by 74%. These findings indicate that the proposed framework effectively balances commercial monetization efficiency with user experience preservation in conversational advertising systems.

Abstract

Conversational advertising aims to deliver useful ads within multi-turn assistant interactions. Unlike conventional query-based advertising, where the user's intent is often expressed in a short standalone query, conversational ads must infer latent commercial intent from the current user query, the assistant response, and dialogue history while also deciding whether an ad would be helpful rather than intrusive. We propose AdsWorldEngine, an agentic framework for conversational advertising. AdsWorldEngine uses an Opportunity Gate to determine whether ads should be shown, an Orchestrator to generate commercial intents, call advertising tools, and construct a top-3 ad slate, and an Evaluator to score delivered ads for offline optimization. The central contribution is an iterative actor-tool training procedure: we first train the Orchestrator with supervised fine-tuning and agentic reinforcement learning, then use high- and low-reward rollouts to construct preference data to train tools. This creates a self-improving loop in which the system learns not only how to use advertising tools, but also how to improve them from rewarded behavior. To support subjective production decisions, we introduce label grounded judgment modeling, which trains judgment models from human labels collected under explicit guidelines. It enriches labels with thinking traces, filters inconsistent rationales through reflection, and further optimizes binary judgments with a cost sensitive GRPO variant that preserves asymmetric reward gaps. Offline, AdsWorldEngine improves diversity by 60% and relevance by 80% over the current production ad delivery system. In an online A/B test, it increases RPM by 22% and ads coverage by 74%.

Problem

• Conversational Advertising
• Latent Commercial Intent
• Ad Intrusiveness
• Multi-turn Interaction

Innovation

• Orchestrator-Tool Coevolution
• Agentic Reinforcement Learning
• Label Grounded Judgment Modeling
• Cost-Sensitive GRPO
• Conversational Advertising