Adaptive$^2$: Adaptive Domain Mining for Fine-grained Domain Adaptation Modeling

2024-12-11

AI Summary

Online advertising systems face significant challenges due to pronounced data distribution shifts across multiple scenarios and suboptimal, manually defined coarse-grained domains. To address these issues, we propose the first end-to-end joint learning framework that simultaneously optimizes fine-grained domain identification and adaptive modeling. Our method introduces a self-supervised domain mining module based on Vector Quantized Variational Autoencoders (VQ-VAE), eliminating reliance on handcrafted domain priors (e.g., ad placements). We further design a shared-specific network architecture to enable both cross-domain knowledge transfer and domain-specific representation learning. Additionally, we integrate self-supervised signals to enhance the robustness of domain discrimination. Extensive experiments on public benchmarks demonstrate substantial improvements over state-of-the-art domain adaptation methods, outperforming single-domain models under comparable FLOPs. The framework has been successfully deployed in Kuaishou’s live-streaming advertising system, validating its commercial efficacy and capability for automated, data-driven domain discovery.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Application Domains: Web
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Advertising systems often face the multi-domain challenge, where data distributions vary significantly across scenarios. Existing domain adaptation methods primarily focus on building domain-adaptive neural networks but often rely on hand-crafted domain information, e.g., advertising placement, which may be sub-optimal. We think that fine-grained"domain"patterns exist that are difficult to hand-craft in online advertisement. Thus, we propose Adaptive$^2$, a novel framework that first learns domains adaptively using a domain mining module by self-supervision and then employs a shared&specific network to model shared and conflicting information. As a practice, we use VQ-VAE as the domain mining module and conduct extensive experiments on public benchmarks. Results show that traditional domain adaptation methods with hand-crafted domains perform no better than single-domain models under fair FLOPS conditions, highlighting the importance of domain definition. In contrast, Adaptive$^2$ outperforms existing approaches, emphasizing the effectiveness of our method and the significance of domain mining. We also deployed Adaptive$^2$ in the live streaming scenario of Kuaishou Advertising System, demonstrating its commercial value and potential for automatic domain identification. To the best of our knowledge, Adaptive$^2$ is the first approach to automatically learn both domain identification and adaptation in online advertising, opening new research directions for this area.

Problem

• Addresses multi-domain data distribution challenges in advertising systems.
• Proposes Adaptive$^2$ for automatic domain mining and adaptation modeling.
• Demonstrates superior performance over traditional domain adaptation methods.

Innovation

• Self-supervised domain mining module
• Shared&specific network modeling
• VQ-VAE for adaptive domain learning