Large Foundation Model for Ads Recommendation

2025-08-20

AI Summary

Existing advertising recommendation methods transfer large-model representations solely via user embeddings, neglecting item and user-item interaction representations, while employing single-granularity transfer and suffering from decoupled upstream/downstream optimization. This paper proposes LFM4Ads—the first full-representation, multi-granularity transfer framework tailored for advertising recommendation—uniquely enabling concurrent transfer of user, item, and user-item interaction representations. It innovatively introduces a layer selection and aggregation mechanism to model interaction representations, complemented by nonlinear adapters, isomorphic interaction modules, and independent retrieval mechanisms to achieve synergistic transfer at feature-, module-, and model-levels. The framework supports industrial-scale deployment with trillion-scale sparse embeddings and terabyte-scale parameters. Deployed across十余 scenarios in Tencent’s advertising system, it boosts platform-wide GMV by 2.45% and generates annual incremental revenue in the hundreds of millions of USD.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Adversarial Agents
• Data Mining & Knowledge Management: Recommender Systems

Application Category

• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Search and Retrieval-Augmented AI: Ad search and search for Web retail
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Online advertising relies on accurate recommendation models, with recent advances using pre-trained large-scale foundation models (LFMs) to capture users' general interests across multiple scenarios and tasks. However, existing methods have critical limitations: they extract and transfer only user representations (URs), ignoring valuable item representations (IRs) and user-item cross representations (CRs); and they simply use a UR as a feature in downstream applications, which fails to bridge upstream-downstream gaps and overlooks more transfer granularities. In this paper, we propose LFM4Ads, an All-Representation Multi-Granularity transfer framework for ads recommendation. It first comprehensively transfers URs, IRs, and CRs, i.e., all available representations in the pre-trained foundation model. To effectively utilize the CRs, it identifies the optimal extraction layer and aggregates them into transferable coarse-grained forms. Furthermore, we enhance the transferability via multi-granularity mechanisms: non-linear adapters for feature-level transfer, an Isomorphic Interaction Module for module-level transfer, and Standalone Retrieval for model-level transfer. LFM4Ads has been successfully deployed in Tencent's industrial-scale advertising platform, processing tens of billions of daily samples while maintaining terabyte-scale model parameters with billions of sparse embedding keys across approximately two thousand features. Since its production deployment in Q4 2024, LFM4Ads has achieved 10+ successful production launches across various advertising scenarios, including primary ones like Weixin Moments and Channels. These launches achieve an overall GMV lift of 2.45% across the entire platform, translating to estimated annual revenue increases in the hundreds of millions of dollars.

Problem

• Transferring all representations from pre-trained models for ads
• Bridging upstream-downstream gaps in recommendation systems
• Enhancing multi-granularity transfer mechanisms for improved performance

Innovation

• Transfers all representations: user, item, cross
• Extracts optimal layer and aggregates cross representations
• Uses multi-granularity transfer mechanisms: feature, module, model