Transferable and Forecastable User Targeting Foundation Model

2024-12-17

arXiv.org

AI Summary

Existing user targeting methods exhibit weak cross-domain generalizability and insufficient real-world predictive capability, limiting their adaptability to diverse industrial scenarios. To address this, we propose FOUND, an industrial-scale foundational model for user targeting. FOUND introduces a unified modeling framework that jointly optimizes cross-domain transferability and behavioral predictability: (1) contrastive pretraining aligns heterogeneous multi-source user data with concise, sentence-level demand inputs; (2) future behavioral text descriptions are incorporated to enhance temporal forecasting; and (3) joint representation learning integrates historical behavior encoding with future behavior generation. Deployed at scale on the Alipay platform, FOUND consistently outperforms state-of-the-art methods across multiple metrics. It enables real-time, high-precision user targeting across heterogeneous business scenarios—including marketing, recommendation, and risk management—demonstrating strong industrial viability and robust generalization.

Technology Category

• Search and Optimization: Other Foundations of Search & Optimization
• Machine Learning: Other Foundations of Machine Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

User targeting, the process of selecting targeted users from a pool of candidates for non-expert marketers, has garnered substantial attention with the advancements in digital marketing. However, existing user targeting methods encounter two significant challenges: (i) Poor cross-domain and cross-scenario transferability and generalization, and (ii) Insufficient forecastability in real-world applications. These limitations hinder their applicability across diverse industrial scenarios. In this work, we propose FOUND, an industrial-grade, transferable, and forecastable user targeting foundation model. To enhance cross-domain transferability, our framework integrates heterogeneous multi-scenario user data, aligning them with one-sentence targeting demand inputs through contrastive pre-training. For improved forecastability, the text description of each user is derived based on anticipated future behaviors, while user representations are constructed from historical information. Experimental results demonstrate that our approach significantly outperforms existing baselines in cross-domain, real-world user targeting scenarios, showcasing the superior capabilities of FOUND. Moreover, our method has been successfully deployed on the Alipay platform and is widely utilized across various scenarios.

Problem

• Enhancing cross-domain transferability in user targeting
• Improving forecastability in real-world applications
• Developing a foundation model for diverse industrial scenarios

Innovation

• Contrastive pre-training enhances transferability
• Future behavior text improves forecastability
• Historical data constructs user representations