Fine-Tuned LLM as a Complementary Predictor Improving Ads System

2026-05-26

AI Summary

This work explores the effective integration of large language models (LLMs) into large-scale advertising recommendation systems. Addressing limitations in conventional candidate generation and ranking modules, the study proposes a novel LLM application paradigm: an open-source LLM, fine-tuned on domain-specific data, serves as a dedicated auxiliary predictor that leverages user profiles and behavioral sequences to forecast potential advertisers. The resulting predictions provide complementary signals for candidate generation and inject informative priors into the ranking module. This approach enables end-to-end optimization across retrieval and ranking stages. Deployed in an industrial advertising system, the method yields significant improvements in offline evaluation metrics and delivers substantial online business gains, demonstrating the practical utility and innovative potential of LLMs in real-world advertising scenarios.

Abstract

Recommendation systems power engagement and monetization across feeds, ads, and short-video platforms, but translating the latest advances in Large Language Models into Recommendation Systems (RecSys) gains remains rare, particularly in advertising and production-scale real-world industry setups. Prior real-world LLM successes typically fall into three buckets: (a) generative retrieval that directly predicts the next items for candidate generation, (b) late-stage re-ranking that uses LLMs, and (c) auxiliary signal enrichment with LLMs. We introduce a complementary paradigm for ads: a fine-tuned open-source LLM used not as a ranker, but as an ads-specific ancillary predictor, forecasting likely advertisers from user profiles and histories. This LLM-driven advertiser prediction augments conventional candidate generation and provides informative priors to downstream ranking. Developed in a large-scale production advertising system, our approach produces substantial offline improvements and measurable online business impact, demonstrating that LLM world knowledge and predictive capacity can be efficiently harnessed. Beyond validating LLMs for ads applications, our results show that targeted ancillary predictions can unlock end-to-end gains across both retrieval and late-stage ranking, offering a practical path to LLM-enhanced recommendation at scale.

Problem

• Large Language Models
• Recommendation Systems
• Advertising
• Candidate Generation
• Auxiliary Prediction

Innovation

• fine-tuned LLM
• complementary predictor
• advertiser prediction
• candidate generation
• recommendation systems