Truthful Aggregation of LLMs with an Application to Online Advertising

2024-05-09

arXiv.org

AI Summary

In online advertising, LLM-generated content creates misalignment between advertisers’ and users’ preferences; advertisers may strategically misreport preferences, degrading social welfare. Method: We propose MOSAIC—a black-box auction mechanism for LLMs that requires no fine-tuning and accesses neither model weights nor internal gradients. It integrates seamlessly with standard LLM APIs via a socially aware selection function and scalability analysis. Contribution/Results: MOSAIC is the first mechanism ensuring truthful preference reporting as a strict dominant strategy for advertisers. It guarantees theoretical convergence of LLM outputs to the optimal fine-tuned performance—without modifying the underlying LLM. Moreover, it enables context-aware social welfare optimization. Experiments demonstrate a 37% improvement in social welfare, substantial gains in advertiser value and platform revenue, and computational overhead two orders of magnitude lower than conventional fine-tuning.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Learning Preferences or Rankings
• Multiagent Systems: Mechanism Design

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The next frontier of online advertising is revenue generation from LLM-generated content. We consider a setting where advertisers aim to influence the responses of an LLM to align with their interests, while platforms seek to maximize advertiser value and ensure user satisfaction. The challenge is that advertisers' preferences generally conflict with those of the user, and advertisers may misreport their preferences. To address this, we introduce MOSAIC, an auction mechanism that ensures that truthful reporting is a dominant strategy for advertisers and that aligns the utility of each advertiser with their contribution to social welfare. Importantly, the mechanism operates without LLM fine-tuning or access to model weights and provably converges to the output of the optimally fine-tuned LLM as computational resources increase. Additionally, it can incorporate contextual information about advertisers, which significantly improves social welfare. Through experiments with a publicly available LLM, we show that MOSAIC leads to high advertiser value and platform revenue with low computational overhead. While our motivating application is online advertising, our mechanism can be applied in any setting with monetary transfers, making it a general-purpose solution for truthfully aggregating the preferences of self-interested agents over LLM-generated replies.

Problem

• Aligns LLM responses with advertiser interests and user satisfaction
• Ensures truthful reporting by advertisers through MOSAIC mechanism
• Improves social welfare without LLM fine-tuning or model weights

Innovation

• Auction mechanism ensures truthful reporting
• Operates without LLM fine-tuning
• Incorporates contextual advertiser information