Large Language Model Simulator for Cold-Start Recommendation

2024-02-14

AI Summary

To address degraded recommendation accuracy caused by cold-start scenarios (e.g., new users or long-tail items) in billion-scale online recommendation systems, this paper proposes ColdLLM—a novel framework leveraging large language models (LLMs) to simulate user interactions with cold items for the first time. It introduces a two-stage coupled funnel architecture that jointly optimizes accuracy and latency, enabling millisecond-level online inference. The method integrates behavioral modeling, lightweight candidate pruning, and real-time serving optimization. Extensive experiments demonstrate significant improvements over state-of-the-art baselines: +3.2% Recall@50 and +2.8% NDCG@10. A two-week A/B test confirms industrial viability, yielding a 12.7% GMV lift for cold-start users. The core contributions are (1) an LLM-driven behavioral simulation paradigm for cold items and (2) a scalable, production-ready cold-start architecture tailored for ultra-large-scale systems.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recommending cold items remains a significant challenge in billion-scale online recommendation systems. While warm items benefit from historical user behaviors, cold items rely solely on content features, limiting their recommendation performance and impacting user experience and revenue. Current models generate synthetic behavioral embeddings from content features but fail to address the core issue: the absence of historical behavior data. To tackle this, we introduce the LLM Simulator framework, which leverages large language models to simulate user interactions for cold items, fundamentally addressing the cold-start problem. However, simply using LLM to traverse all users can introduce significant complexity in billion-scale systems. To manage the computational complexity, we propose a coupled funnel ColdLLM framework for online recommendation. ColdLLM efficiently reduces the number of candidate users from billions to hundreds using a trained coupled filter, allowing the LLM to operate efficiently and effectively on the filtered set. Extensive experiments show that ColdLLM significantly surpasses baselines in cold-start recommendations, including Recall and NDCG metrics. A two-week A/B test also validates that ColdLLM can effectively increase the cold-start period GMV.

Problem

• Cold Start Problem
• Recommendation Accuracy
• User Experience

Innovation

• LLM Simulator
• ColdLLM Framework
• Non-popular Item Recommendation