A Soft-partitioned Semi-supervised Collaborative Transfer Learning Approach for Multi-Domain Recommendation

2025-11-03

AI Summary

Addressing the dual challenges of dominant-domain data overwhelming and overfitting in sparse non-dominant domains within multi-domain recommendation, this paper proposes a soft-partitioning semi-supervised collaborative transfer learning framework. Methodologically, it employs a shared-specific network architecture that jointly integrates collaborative transfer learning and semi-supervised training. Its key contributions are: (1) a dynamic parameter generation mechanism for soft domain partitioning, which mitigates inter-domain interference; and (2) a weighted pseudo-labeling strategy for semi-supervised transfer, enhancing generalization in data-sparse domains. Extensive offline and online experiments demonstrate consistent performance gains across multiple domains: GMV improves by 0.54%–2.90%, and CTR increases by 0.22%–1.69%. The framework effectively achieves balanced multi-domain performance and improved robustness.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Application Domains: Humanities & Computational Social Science
• Data Mining & Knowledge Management: Recommender Systems

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

In industrial practice, Multi-domain Recommendation (MDR) plays a crucial role. Shared-specific architectures are widely used in industrial solutions to capture shared and unique attributes via shared and specific parameters. However, with imbalanced data across different domains, these models face two key issues: (1) Overwhelming: Dominant domain data skews model performance, neglecting non-dominant domains. (2) Overfitting: Sparse data in non-dominant domains leads to overfitting in specific parameters. To tackle these challenges, we propose Soft-partitioned Semi-supervised Collaborative Transfer Learning (SSCTL) for multi-domain recommendation. SSCTL generates dynamic parameters to address the overwhelming issue, thus shifting focus towards samples from non-dominant domains. To combat overfitting, it leverages pseudo-labels with weights from dominant domain instances to enhance non-dominant domain data. We conduct comprehensive experiments, both online and offline, to validate the efficacy of our proposed method. Online tests yielded significant improvements across various domains, with increases in GMV ranging from 0.54% to 2.90% and enhancements in CTR ranging from 0.22% to 1.69%.

Problem

• Addresses data imbalance skewing multi-domain recommendation performance
• Mitigates overfitting in sparse non-dominant domain recommendation data
• Generates dynamic parameters to prioritize non-dominant domain samples

Innovation

• Dynamic parameters address overwhelming from dominant domains
• Pseudo-labels with weights enhance sparse non-dominant domain data
• Soft-partitioned semi-supervised transfer learning improves multi-domain recommendation