Entity Representation Learning Through Onsite-Offsite Graph for Pinterset Ads

2025-08-04

AI Summary

This paper addresses the challenge of effectively leveraging off-site user conversion signals in advertising ranking models. To this end, it proposes a large-scale heterogeneous graph modeling framework that jointly encodes on-site interactions and off-site conversion behaviors. The method introduces TransRA, a novel knowledge graph embedding (KGE) model featuring an anchor mechanism to mitigate data sparsity and an attention-driven KGE fine-tuning strategy to enhance cross-domain behavioral semantics modeling. Additionally, it integrates graph neural networks, a Large ID Embedding Table, and attention mechanisms for efficient entity representation learning. Deployed in Pinterest’s advertising system, the approach yields a 2.69% lift in click-through rate (CTR) and a 1.34% reduction in cost-per-click (CPC), significantly improving both CTR and conversion rate prediction accuracy. These results empirically validate the effectiveness and practicality of joint cross-domain behavioral modeling for large-scale ad ranking.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Application Domains: Humanities & Computational Social Science

Application Category

• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Graph Neural Networks (GNN) have been extensively applied to industry recommendation systems, as seen in models like GraphSagecite{GraphSage}, TwHIMcite{TwHIM}, LiGNNcite{LiGNN} etc. In these works, graphs were constructed based on users' activities on the platforms, and various graph models were developed to effectively learn node embeddings. In addition to users' onsite activities, their offsite conversions are crucial for Ads models to capture their shopping interest. To better leverage offsite conversion data and explore the connection between onsite and offsite activities, we constructed a large-scale heterogeneous graph based on users' onsite ad interactions and opt-in offsite conversion activities. Furthermore, we introduced TransRA (TransRcite{TransR} with Anchors), a novel Knowledge Graph Embedding (KGE) model, to more efficiently integrate graph embeddings into Ads ranking models. However, our Ads ranking models initially struggled to directly incorporate Knowledge Graph Embeddings (KGE), and only modest gains were observed during offline experiments. To address this challenge, we employed the Large ID Embedding Table technique and innovated an attention based KGE finetuning approach within the Ads ranking models. As a result, we observed a significant AUC lift in Click-Through Rate (CTR) and Conversion Rate (CVR) prediction models. Moreover, this framework has been deployed in Pinterest's Ads Engagement Model and contributed to $2.69%$ CTR lift and $1.34%$ CPC reduction. We believe the techniques presented in this paper can be leveraged by other large-scale industrial models.

Problem

• Integrating offsite conversion data with onsite user activities for ads
• Improving Ads ranking models with Knowledge Graph Embeddings (KGE)
• Enhancing CTR and CVR predictions through innovative embedding techniques

Innovation

• Constructed onsite-offsite heterogeneous graph for ads
• Introduced TransRA for efficient KGE integration
• Used Large ID Embedding Table and attention finetuning