Enhancing User Interest based on Stream Clustering and Memory Networks in Large-Scale Recommender Systems

2024-05-21

Social Science Research Network

AI Summary

To address the challenge of sparse user purchase histories in large-scale recommender systems—which impedes accurate interest modeling and degrades recommendation performance—this paper proposes an end-to-end User Interest Enhancement (UIE) framework. UIE is the first approach to jointly leverage streaming clustering and an external memory network for dynamically modeling the evolution of user interests. It employs a dual-path mechanism to simultaneously optimize representations for sparse users and improve matching for long-tail items, and integrates seamlessly as a lightweight plug-in into existing ranking models. By unifying user profile and behavioral sequence modeling, UIE generates personalized enhanced embedding vectors. Deployed at scale in an industrial setting with tens of millions of daily active users, UIE achieves significant improvements in AUC and CTR; notably, recommendation performance for sparse users improves by over 12%.

Technology Category

• Data Mining & Knowledge Management: Recommender Systems
• Machine Learning: Learning Preferences or Rankings
• Humans and AI: Intelligent User Interfaces

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Recommender Systems (RSs) provide personalized recommendation service based on user interest, which are widely used in various platforms. However, there are lots of users with sparse interest due to lacking consumption behaviors, which leads to poor recommendation results for them. This problem is widespread in large-scale RSs and is particularly difficult to address. To solve this problem, we propose a novel solution named User Interest Enhancement (UIE) which enhances user interest including user profile and user history behavior sequences using the enhancement vectors and personalized enhancement vector generated based on stream clustering and memory networks from different perspectives. UIE not only remarkably improves model performance on the users with sparse interest but also significantly enhance model performance on other users. UIE is an end-to-end solution which is easy to be implemented based on ranking model. Moreover, we expand our solution and apply similar methods to long-tail items, which also achieves excellent improvement. Furthermore, we conduct extensive offline and online experiments in a large-scale industrial RS. The results demonstrate that our model outperforms other models remarkably, especially for the users with sparse interest. Until now, UIE has been fully deployed in multiple large-scale RSs and achieved remarkable improvements.

Problem

• Recommendation Systems
• Sparse User Data
• Cold Start Problem

Innovation

• User Interest Enhancement
• Sparse Data Handling
• Recommendation Quality Improvement