TeamCMU at Touché: Adversarial Co-Evolution for Advertisement Integration and Detection in Conversational Search

2025-07-01

AI Summary

In generative conversational search, LLM+RAG systems naturally embed advertisements into responses, blurring the boundary between informational content and commercial promotion—thereby undermining transparency and user trust. To address this, we propose a modular advertising management framework centered on an adversarial co-evolution mechanism between an ad rewriter and an ad detector classifier. The classifier is trained via synthetic data generation and curriculum learning to enhance robustness, while the rewriter is optimized via supervised fine-tuning and best-of-N sampling to improve ad stealthiness. This end-to-end pipeline integrates RAG, synthetic data generation, fine-tuning, and sampling techniques to preserve advertising utility while significantly reducing detectability. Experiments demonstrate high detection accuracy of the classifier and markedly more natural, less intrusive ad insertion—achieving, for the first time, controllable ad integration that jointly optimizes commercial value and user experience in generative search.

Technology Category

• Search and Optimization: Adversarial Search
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Ad search and search for Web retail
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

As conversational search engines increasingly adopt generation-based paradigms powered by Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG), the integration of advertisements into generated responses presents both commercial opportunities and challenges for user experience. Unlike traditional search, where advertisements are clearly delineated, generative systems blur the boundary between informational content and promotional material, raising concerns around transparency and trust. In this work, we propose a modular pipeline for advertisement management in RAG-based conversational systems, consisting of an ad-rewriter for seamless ad integration and a robust ad-classifier for detection. We leverage synthetic data to train high-performing classifiers, which are then used to guide two complementary ad-integration strategies: supervised fine-tuning of the ad-rewriter and a best-of-N sampling approach that selects the least detectable ad-integrated response among multiple candidates. Our evaluation focuses on two core questions: the effectiveness of ad classifiers in detecting diverse ad integration strategies, and the training methods that best support coherent, minimally intrusive ad insertion. Experimental results show that our ad-classifier, trained on synthetic advertisement data inspired by marketing strategies and enhanced through curriculum learning, achieves robust detection performance. Additionally, we demonstrate that classifier-guided optimization, through both fine-tuning and best-of-N sampling, significantly improves ad stealth, enabling more seamless integration. These findings contribute an adversarial co-evolution framework for developing more sophisticated ad-aware generative search systems and robust ad classifiers.

Problem

• Managing ad integration in conversational search systems
• Detecting ads in LLM-generated responses effectively
• Balancing ad stealth and user experience

Innovation

• Modular pipeline for ad management in RAG systems
• Synthetic data trains robust ad-classifiers
• Classifier-guided optimization enhances ad stealth